See what our 2021 looked like. Numbers grow, the goal stays the same – to deliver a stellar client service to you and the best possible online experience to your users. Thank you for trusting in us. 

Capturing a moment and sharing it with thousands of people worldwide in under a second is undoubtedly cool, but what are the drawbacks? In this blog post, I will provide an overview of the setup, tweaks, and compromises necessary to achieve low latency when streaming over HTTP. Hopefully, this blog post can give you an insight into how low-latency HTTP live streaming pipelines are designed.

Since there aren't any official definitions for the term "low latency," you can stumble across various meanings. In this blog, we discuss "glass-to-glass latency," which means how long it takes between something happening in front of the camera and users seeing it on their screens. We will also label anything under 5 seconds "low latency" and anything under 1 second "ultra-low latency."

For reference, regular live streaming over HTTP latency is usually 20 to 40 seconds, and digital TV broadcasts take around 4 to 6 seconds to reach the audience.

There are a few use cases where low latency might be absolutely crucial. For example, streamers on social media need to be able to react promptly to their audience in chat. Keeping the stream engaging is impossible if there's a 30-second latency in the communication channel. The same goes for major sports events. You want to see the goal yourself before you hear your neighbor cheering.

HTTP is a request-response protocol that currently underlies all web technologies. It's been around for decades and is, therefore, highly supported and has very mature client and server implementations. This is the main advantage over newer solutions such as WebRTC, which, although initially meant for video calls, is also sometimes utilized for low-latency streaming.

Using HTTP also gets you DVR and VOD by default, meaning you can use the same media files you transcoded for low latency for VOD later. You can also easily fall back to normal latency for clients that don't support low latency without a need to re-encode the media files. This is especially useful for social media platforms, where you have a lot of long-tail broadcasts with 0-10 users, so it's extremely helpful to reuse the files the way they are. In general, if you have a few really popular channels, like traditional broadcasts, re-encoding VOD to achieve better compression ratios and bandwidth is very good practice.

HTTP live streaming is powered by two main competing protocols.

At their core, they operate on the same principle. They split up the original video into video segments, usually a couple of seconds in duration, and then create an index manifest. The index manifest contains general stream info, but most importantly, it lists various video segments. The client then downloads the index and can choose which video segment to download to continue in a seamless stream. It also supports adaptive bitrate, providing the segments in multiple bitrates and resolutions.

In the case of VOD, the index is static, so the client needs to download it only once. In the case of live streaming, you need to keep downloading the index in a loop so you always have the most recent version.

Let's take a look at a live-streaming pipeline.

Source is the device that captures a video.

Network RTT (Round-Trip Time) refers to the time necessary for the file to be sent and acknowledged by ingest.

Ingest receives and formats the video so it can be handled by a transcoder.

Transcoder converts the files to multiple resolutions and bitrates so they're ready for an adaptive bitrate.

Packager segments the video into small chunks of data and creates the index manifest.

Origin (storage) is where the files are waiting to be served to the Client.

CDN serves the data optimally from the origin or from the edges where the content is cached.

Client is the device the end-user uses to watch the stream, fetch the index manifests, and download the created segments. 

Read-ahead buffer is the pre-load of the video files used to mitigate potential disruptions to the stream.

As the diagram shows, the most significant sources of latency are segment duration, read-ahead buffer, transcoder, and source. I will cover how to optimize these steps in an order that's hopefully most understandable for readers new to this topic. Optimization of the source will be skipped since that's usually determined by the hardware and is out of the scope of this article.

Whenever the packager creates a segment, it uploads it to the origin, and only then can the client fetch it. Once that's done, the whole process repeats itself. If the segments are 5 seconds long, that's another 5-second latency just from this one loop.

The simple solution would be to decrease the duration of the segment, which works but comes with its own problems. You will face two big issues once you start going under two-second segments.

HTTP overhead – Since you create more requests, you must also process additional data beyond the actual video files.

Codec inefficiencies – Keyframes take a lot of space and must be present in every segment. If you need them too frequently, less quality is left for other frames, and your compression ratio decreases.

Luckily, MP4 and H.264/H.265 codec don't require the entire segment to be completed before it can be decoded and played. Even better, HTTP supports chunk transfer, which lets you transfer files before they're fully created or their file size is known.

The outcome of this combination is that once the packager starts processing the file, you can simultaneously start uploading it to the origin. At the same moment, the client can also start fetching, decoding, and playing it nearly instantly. Suddenly, we're reducing the delay from 5-10 s to 50-100 ms since it's limited just by network round-trip time.

The issue that persists is fetching the index, which still needs to happen continuously.

For MPEG-DASH, we can use Low latency mode, which supports chunk transfer and a segmentation template. The template contains static information about the segments, such as stream start time, segment duration, segment naming, and their path template. This information doesn't need to change every segment. Using this, the client only fetches the index once and then can locally calculate which segment to download to continue the seamless video stream.

The situation is more complex for HLS. For years, we used community-developed low-latency HLS, which didn't solve the index issue directly but made segments fully HLS and DASH compatible through a common media application format (CMAF). It also supported having the same files as DASH over chunk transfer.

This was until 2018 when Apple made the low-latency HLS incompatible with this community solution and presented its own low-latency HLS. It wasn't very welcomed by the community. It doesn't rely on chunk transfer to transfer complete segments in one request but tries to use separate partial segments. The standard isn't yet stabilized, and we're currently on draft 16 from November 2024, which expires in May 2025.

Currently, you can use low-latency DASH on everything except iOS browsers. This is a bit annoying since even iPadOS is supported, so it really affects only iPhones.

These are your options for reducing the latency of index fetching. If you set it up correctly, you no longer wait 5 to 10 seconds for the index; instead, you push it down to something like 10 milliseconds.

Another thing to consider is the compatibility of your infrastructure. Essentially, everything after the packager needs support for chunk transfer or partial segment serving. This might be surprising, but it's less common than you might think. Many storages, for example, AWS S3 object storage, don't support uploading before you know the file size (outside of Express One Zones). Since we're uploading the file as it's being created, we cannot know the file size and, therefore, use it this way.

Nginx is at the core of web servers and many CDNs and it also doesn't support caching files that are not yet complete, at least in its native form. So you need to have a custom implementation of it, like we did, to be able to optimize the stream for low latency.

Overall, consider your whole infrastructure before you start implementing any technology so that you know what you're dealing with.

Read-ahead buffer pre-downloads segments to make the stream resilient to short network disruptions. So, the latency created is entirely intentional and usually very helpful. If you set it to 10 seconds, you have enough time to download the missing segment or switch to a lower bitrate. This can have a huge impact on the continuity and flow of the stream. Since we're making other compromises for low latency, what happens if we get rid of it?

Well, besides the instabilities, you will face another major problem. The buffer is used to detect when to switch bitrates. If the buffer is full, it may switch to a higher bitrate because your network obviously has no issues getting the files in time. But if your buffer is running low, it's a sign that dropping to a lower bitrate is necessary.

Without the buffer, downloading files as they're being created and judging how fast they're downloaded is much harder. Since you're downloading a 10-second file as it's being created, it'll always take at least 10 seconds to download, so you don't know if your network would be able to download a higher bitrate in time.

There is ongoing PhD research (L2A-LL, LoL+) on this topic, so the solution is still being worked on. What's important is that every time you decide to make changes to the read-ahead buffer, you need to adjust the bitrate ladder accordingly.

If you removed (or significantly lowered) the buffer and the user encounters a network problem during a live stream, you still have some options to mitigate the problem. If they download the data later, you can simply skip the missing segment and go back to live, but that would seriously hamper the viewer experience. Instead, you can slightly adjust the playback speed by a few percent. For example, a 5% faster video is practically unnoticeable to the viewer, but it helps you to catch up to the live edge. The same goes for slightly slower playback, which can help you create at least a small buffer if your network is unstable.

This can be done on the fly on a per-user basis, meaning that you can leave a tiny buffer for someone at home with stable wired internet and a bigger buffer for someone who is on the go with mobile internet. More can be done on the network side, but that would be a topic for another blog post, so let's just move further.

Optimizing transcoding is quite complex and relatively difficult on its own. Chances are, if you're reading this blog post, you don't want to develop your own transcoder. Luckily, using existing "off-the-shelf" transcoders is sufficiently efficient. They typically have a low latency switch, which turns off features that increase the delay, like the look-ahead buffer or B frames. Overall, getting under 15 ms is relatively easy with currently available transcoders.

With the methods mentioned in this blog post, you can easily achieve under 5-second or even 1-second latency with ordinary HTTP. Just remember that there will always be a tradeoff between stability and latency, but the technology itself is ready and well-tested. The drawbacks we covered are also not an issue with HTTP; they're fundamental to the nature of the service. So consider what is more important for your broadcast.

Thank you for reading this far, and good luck with adjusting your pipeline. If this short blog sparked your interest in the topic, I consider that a win. To get deeper into it, I recommend the paper Toward One-Second Latency and RFCs 8673, 9317. And of course, I'm always happy to chat with you regarding anything LLLS, streaming, or CDN-related. Just hit me up!

In this blog post, I will provide an overview of the setup, tweaks, and compromises necessary to achieve low latency when streaming over HTTP.

In the ever-evolving landscape of content delivery networks, we understand the importance of giving you, our clients, the right tools you need to be more autonomous in your operations. That’s why we’re eager to introduce a significant advancement that puts the power squarely in your hands: CDN77 Edge Computing.

Building upon standard solutions, this platform allows you to customize online interactions according to your business and technical goals, ensuring high performance and security. We aim to empower you with complete control over how our edge servers transform, cache, and handle HTTP requests.

This blog post will explore the essence of CDN77 Edge Computing and the challenges we overcame during its development. We know that for many clients this is a game changer, so we’re launching a beta access program. You can already reach out to your account manager or sales to apply for beta access. We would love to hear your feedback or feature suggestions for your specific use case.

As we specialize in content delivery, we depend heavily on our infrastructure. We operate thousands of edge servers running across the globe. This infrastructure is supported by a 130 Tbps network with a global private backbone.

It’s no secret that our edge servers are powered by our fork of Nginx enhanced with OpenResty. This dynamic combination empowers our servers to store and deliver content efficiently. It offers advanced customization capabilities through real-time manipulation of HTTP requests and responses, ensuring optimal performance and tailored content delivery for every user.

Through our client panel or an extensive API, it’s possible to set up various features, rules and basic request manipulation. These configurations are then propagated to our edge servers and executed when needed. These configs’ technical implementation usually combines Lua scripts and Nginx configurations.

For customers who need more advanced or tailored configurations that aren’t generally available from our API, we have a Support Team that can configure almost any request or routing transformation.

While our API covers most of the common use cases of HTTP request transformation, we wanted to offer more. Our customers know what is best for them and should be allowed to manipulate their requests however they want, whenever they want, in a secure and isolated manner. Essentially, we wanted to give them full autonomy in what they can do themselves and let them see and debug routing changes instantly.

Our focus in designing the Edge Computing platform was to be able to transform requests securely and dynamically. We also needed to load new modifications and plugins dynamically on the fly without influencing or slowing down anything else running on our edge servers. Additionally, we wanted our plugins to work and seamlessly integrate with Nginx, allowing us to adjust virtually any internal behavior as needed.

Another goal was to be able to express any business logic that the client would ever need. Therefore, we want to be able to execute actual code and not be limited by basic grammar and expression. However, this decision posed a challenge: How to execute untrusted code safely without compromising the security of our servers?

Running untrusted code in a trusted environment is hard. Ensuring proper isolation and sandboxing mechanisms to prevent unauthorized access to system resources, like files, network, or customer data, demands meticulous attention to detail and can be complex to implement effectively. Moreover, we need to be running untrusted code fast.

While designing our platform, we considered using isolated processes, microVMs or even containers. We found isolated processes not safe enough for our use case. MicroVMs are secure but bring other challenges like long cold starts and slow communication between plugins and our Nginx. Running containers might help with the cold starts and communication overhead, but it's still light years behind our final choice.

The answer for our use case lies in the binary world - WebAssembly.

WebAssembly (Wasm) is an ideal choice for executing untrusted code on servers due to its sandboxed and efficient design. Wasm provides a low-level binary instruction format that can be executed at near-native speed, ensuring high performance for diverse workloads. The sandboxing mechanism of Wasm provides a secure execution environment that isolates untrusted code from the host system, mitigating potential security risks and preventing unauthorized access to resources.

Wasm’s platform-independent architecture enables code portability across different servers and platforms. Combining that with performance and security, Wasm ensures that untrusted code can be executed reliably and efficiently within a controlled sandbox virtually anywhere.

As a portable format, one only needs a Wasm runtime to execute Wasm plugins. After extensive research, we decided to use Wasmtime as our runtime. It is written in Rust (❤️), with SDK in C, battle-tested, and can be embedded in an already-running process.

One of the many metrics used to measure the performance of a CDN is latency and response time. While building our Edge Computing platform, the main objective was to keep these metrics down and avoid increasing the general infrastructure load. To do that, we decided to run Edge Computing plugins as close as possible to the infrastructure edge, and Nginx is the first place that can understand L7 data and process it. That’s one of the reasons why we decided to embed Wasmtime directly into our Nginx. Nginx Workers processes can now load and execute Wasm plugins and utilize the full power of Wasmtime while keeping the communication overhead between the plugin and the Nginx close to zero.

These are just our examples. You can run virtually any logic that you and your business need. Feel free to utilize the power of thousands of our edge servers and their proximity to the end users.

Let’s explore some examples in Rust. These are the plugins or their variants we’ve been using and deploying to production for some time.

As we journey into the future of web services and content delivery, CDN77’s new Edge Computing platform is set to evolve and expand, bringing more power and flexibility to developers and businesses alike. With a strong foundation already in place, the following features are under active development and will be available soon to the beta testers.

Because any generic computing platform needs to be able to persist data, we’re working on distributed, eventually consistent key-value storage and distributed SQLite database. Persistent distributed storage is currently our priority number one.

We currently have a feature-complete SDK for Rust, Golang, and AssemblyScript. Our JavaScript and Typescript SDKs are still in their infancy, but we plan to open and publish them shortly. Eventually, we plan to deliver even more SDKs for Wasm-enabled languages to make it easy for developers to deploy to our platform using languages they’re familiar with.

While our Edge Computing platform is designed to handle incoming requests to our CDN, we plan to expand its capabilities to enable the dispatch of HTTP requests in the future.

Given our extensive global infrastructure, we aim to provide even greater utility to our customers than is currently possible. One of the ways we're planning to do that is to provide support for server-side rendering frameworks such as Next.js, Nuxt, or SvelteKit.

We hope you share our excitement about the expansive new possibilities opened up by CDN77 Edge Computing. We've been heavily testing both internally and with selected customers for almost a year. We served billions of requests and petabytes of traffic through this platform. In fact, this very site was served to you by one of our edge computing plugins.

We'd love to hear your feedback and suggestions, so do not hesitate to contact us and tell us how you would like to utilize this platform.

We aim to empower you with complete control over how our edge servers transform, cache, and handle HTTP requests. Read our blog!

We aim to empower you with complete control over how our edge servers transform, cache, and handle HTTP requests.

Continuous improvement of product and service aspects is a standard for most tech enterprises; pricing, however, often remains overlooked. Yet this is exactly what sets market leaders apart – their ability to quickly adapt and respond to client’s changing business needs.

At CDN77, all of our clients now benefit from having overage fees set at the unit price of their CDN traffic plan. What’s more is that they enjoy the freedom to easily upgrade or downgrade their plans on a monthly basis. 

This month we've accomplished our long-term goal – a pricing reform where all of our clients have their overage (additional data that exceeds your subscribed traffic volume) fees equal to the unit price of their chosen plan.

When our clients consume more traffic than they subscribe to, we don’t see that as data excess, we would rather see that as growth that we’re here to support.

Let’s see an example. Imagine you’re subscribed to a 3,000TB Plan at $6,600 per month. This means your unit price is $2.2/TB ($6,600/3,000TB). When you transfer more than 3,000TB of traffic in a given month – let’s say 3,500TB – the additional 500TB will be charged at the same rate ($2.2/TB) as the previous 3,000TB. So, your final payment would be $7,700 ($2.2 * 3,500TB).

Over the course of the past 12 months we applied this pricing model to our Custom and High Volume plans. Preset Monthly Plans adjustment marked the end of the transition process this summer, which effectively means that the overage fees are now capped at the unit price for all of CDN77's plans.

Internet usage has never been predictable. The 48% increase in global internet traffic during the 2020 pandemic is a powerful demonstration of traffic volatility. 

At CDN77, we reckon with changes in websites’ and apps’ popularity, traffic seasonality as well as sudden peaks in visits. 

While our 70Tbps+ network absorbs any unexpected traffic surges from the capacity point of view, we make sure to reflect traffic unpredictability also in pricing.

In practice it means that our clients have the freedom to flexibly upgrade or downgrade their traffic plan on a monthly basis according to their actual needs. With no rigid commitments in place, you get the optimal pricing solution, whether you’re experiencing rapid growth or going through a lower traffic span.

For example, you might need a 800 TB/month plan for January, then you can downgrade to 550 TB/month in February and upgrade to 3,500 TB/month in March – if that's how your traffic consumption works, it’s no problem for us.

The combination of flexible plan switching and flat rates for overage traffic makes CDN77 a partner to rely on in all stages of your business.

Adjust Your Plan According to Current Traffic Needs

We’re committed to finding the best fitting solution for all of our clients by consulting their individual use-cases, on both technical and pricing levels. If you’d like to learn more about our services, reach out to our Client Solutions team at clientsolutions@cdn77.com.

With CDN77 you now benefit from flat overages and enjoy the flexibility to change your traffic plans on a monthly basis. Read for more details.

High Overage Fees & Rigid Commitments are Passé at CDN77