Cloudinary Blog

Video Optimization, Part II: Multi-Codec Adaptive Bitrate Streaming

Multi Codec Adaptive Bitrate Streaming

In Part I of this series, we discussed the optimal way to deliver progressive video streams, taking advantage of modern, efficient codecs. That approach works great for short-form videos (under 20 seconds) and for videos that are displayed at a low resolution (such as ads and previews). But what if you're delivering videos that are longer than 20 seconds for a higher-resolution experience? You can certainly still deliver them as a single file (progressive streaming), but you might run into issues, such as buffering or too high a resolution.

As sharp as modern mobile-device screens have become, they can't display all the pixels in high-resolution videos (above 1,920 pixels). Hence, sending down a video too large for the screen is not only a waste of resources and increased bandwidth costs on your part, but also a waste of your users' bandwidth, rendering buffering a likely occurrence.

How can we combine modern codecs with Adaptive Bitrate Streaming to deliver a lightweight and optimal experience to all users, automatically?

Adaptive Bitrate Streaming

Adaptive Bitrate Streaming

A good solution for those problems is Adaptive Bitrate Streaming (ABS). While on YouTube, Netflix, Hulu, or a similar service, if you notice picture quality changing from good to pixelated or vice versa, you've experienced ABS. When encountering slow network conditions, instead of buffering your video, ABS switches dynamically to a lower-bitrate (lower-quality) version of the video, something the connection speed can handle.

ABS also takes into account the available resolution of the viewer. So, if you're viewing the video on an iPhone X, ABS does not deliver the 4K version even if sufficient bandwidth is available to handle such a large file because there are not enough pixels in the display to take advantage of the higher-resolution version. The magic here is that all that happens automatically; you just need to configure the video player with the relevant options.

For a more in-depth discussion of ABS, see the post Plug-and-Play Adaptive Bitrate Streaming With HLS and MPEG-DASH.

Definition of Bitrate

Before diving into how to deliver an optimal adaptive experience, let's talk about the main concept behind it. Bitrate is simply a measure of how many bits it takes to encode a one-second video. Thus, a 10-second video with a 10MB file size has a bitrate of 8 mbps (1 megabyte per second * 8 megabits per megabyte). How can we change the value of the bitrate?

Bitrate = total pixels/second * bytes/pixels

The above equation shows the different levers you can pull to change the bitrate. To achieve a lower bitrate, you can reduce the total number of pixels by reducing the resolution of the video, sampling fewer frames per second, or increasing pixel sharing across frames. You can also reduce the bitrate by compressing the bytes per pixel by lowering the video quality, akin to how you would do that for an image. Part III of this series will suggest the best parameters to adjust for different pieces of content.


In Part I, we discussed modern codecs, such as HEVC (h265) and VP9. These codecs render sharing pixels across frames more efficient, encoding the key frames while maintaining the visual fidelity of videos. In fact, they are about 30-50 percent more efficient than the h264 standard. Currently, however, HEVC and VP9 do not enjoy universal browser or device support.

Not to worry. As described in Part I, you can present your video player with multiple codec options in the adaptive bitrate manifest, which would then choose the first codec it knows how to decode. In fact, about 85 percent of Internet traffic uses one of the modern codecs, meaning that your total bandwidth would be reduced by anywhere between 25 to 43 percent. You can easily calculate how cost-effective this approach would be for you.

Creation of Streaming Profiles for Your Video File

To set up codec options, first create the "streaming profiles" to build the manifests. Think of a streaming profile as a list of all the variations you want to create for ABS of your videos. Here, you can define the aspect ratio, the resolutions, the bitrates, and any additional Cloudinary transformations you’d want to perform to the videos, such as watermarks, subtitles, other text, and splicing in other videos.

Create three streaming profiles, one for each codec you want to support: VP9, H265, and H264. Before doing that, be aware that you have two options:

Option 1: Deliver the same visual experience at less bandwidth cost.

Since H265 and VP9 are more efficient, you can get the same picture quality for a lower bitrate. If saving bandwidth costs is your primary concern, this is probably the way to go.

Option 2: Deliver a superior visual experience for the same bandwidth cost.

If providing your users with a superior visual experience is your primary objective, create a streaming profile with higher bitrates than you would in Option 1 for your H265 and VP9 profiles. That means that at a given bandwidth availability, users whose devices support those modern codecs will get a better visual experience.

Part III will address other options. For now, keep in mind that the content of your video could also affect your decision. For instance, for sport or action content, you might want to maintain a higher bitrate at lower resolutions.

Creating Bitrate Profiles

Before creating the profiles for Option 1, which doesn’t strictly preclude some of the benefits in Option 2, see the Cloudinary API code for creating a streaming profile for H264:

Creating Custom Streaming Profiles

    display_name: "16x9 H264 Codec",
    representations: [
    {transformation:{crop: "pad", width: 1920, aspect_ratio: "16:9", bit_rate: "8500k"}},
    {transformation:{crop: "pad", width: 1280, aspect_ratio: "16:9", bit_rate: "5500k"}},
    {transformation:{crop: "pad", width: 960, aspect_ratio: "16:9", bit_rate: "3500k"}},
    {transformation:{crop: "pad", width: 640, aspect_ratio: "16:9", bit_rate: "2000k"}},
    {transformation:{crop: "pad", width: 480, aspect_ratio: "16:9", bit_rate: "800k"}}
  ] },
  function(error, result){console.log(result);});

Copy that code, paste it into another version, and make the necessary edits for H265, as follows:

    display_name: "16x9 H265 Codec",
    representations: [
    {transformation:{crop: "pad", video_codec: "h265", width: 1920, aspect_ratio: "16:9", bit_rate: "5500k"}},
    {transformation:{crop: "pad", video_codec: "h265", width: 1280, aspect_ratio: "16:9", bit_rate: "3500k"}},
    {transformation:{crop: "pad", video_codec: "h265", width: 960, aspect_ratio: "16:9", bit_rate: "2000k"}},
    {transformation:{crop: "pad", video_codec: "h265", width: 640, aspect_ratio: "16:9", bit_rate: "800k"}},
    {transformation:{crop: "pad", video_codec: "h265", width: 480, aspect_ratio: "16:9", bit_rate: "500k"}}
  ] },
  function(error, result){console.log(result);});

Notice the lower bitrates on H265. This profile is meant to maintain the visual fidelity of the H264 version for less bandwidth.

Now create a profile for VP9 by using the same bitrates you did for H265:

    display_name: "16x9 vp9 Codec",
    representations: [
    {transformation:{crop: "pad", video_codec: "vp9", width: 1920, aspect_ratio: "16:9", bit_rate: "5500k"}},
    {transformation:{crop: "pad", video_codec: "vp9", width: 1280, aspect_ratio: "16:9", bit_rate: "3500k"}},
    {transformation:{crop: "pad", video_codec: "vp9", width: 960, aspect_ratio: "16:9", bit_rate: "2000k"}},
    {transformation:{crop: "pad", video_codec: "vp9", width: 640, aspect_ratio: "16:9", bit_rate: "800k"}},
    {transformation:{crop: "pad", video_codec: "vp9", width: 480, aspect_ratio: "16:9", bit_rate: "500k"}}
  ] },
  function(error, result){console.log(result);});

The above bitrates are meant to be an example only. Though they should work fine in most cases, be sure to test them out and tweak the bitrates to your own standards. Keep in mind that a given bitrate at a constant resolution would result in a video of higher visual quality in VP9 and H265 than in H264. Part III will explore this topic further.

Now create all those versions upon upload of a new video, as follows:

CAUTION: This approach will result in a LOT of transformations--something to take into account because video encoding is expensive. Some of this cost might be offset by the decrease in consumed bandwidth, maybe all of it. You might even save money (sometimes a lot!) doing it this way, depending on the ratio of bandwidth consumption to the number of videos you need encoded. Just be aware of that overhead before encoding your entire Cloudinary library this way. Doing so does not make sense for everyone. As stated earlier, by using this method to reduce bandwidth, you can expect a 25 to 43 percent reduction in total outgoing video bandwidth. With Cloudinary, 1 TB of outgoing bandwidth is equal in cost to one million transformations. You can easily calculate the total cost or cost savings you will see.

Uploading Videos

Next, upload a new video and create the various versions. The above example would read like this:

cloudinary.v2.uploader.upload('video-file.mp4', {resource_type: 'video', 
    eager: [{format: 'm3u8', streaming_profile: '16x9_h264'},
      {format: 'm3u8', streaming_profile: '16x9_h265'},
      {format: 'mpd', streaming_profile: '16x9_vp9'}
    eager_async: true,
    eager_notification_url: 'https://mysite/notify_endpoint', public_id:'yourPublicId'}, function(result) {console.log(result); });

Replace all the variables in bold font with the actual values. When your files are ready, you’ll get an async response to the endpoint you listed under ‘eagernotificationurl`. Remember that you’re creating a lot of files here and, even though VP9 and H265 codecs are extremely efficient consumers of bandwidth, they take about twice as long to encode than H264.

When the upload is complete, you will see the following three manifests, which you’ll need for the next step:

Now you need to decide which manifest to pick for each visitor. Thankfully, David Walsh has already figured it out for us!

Check for H265 support first and then VP9 support. If neither of them works, fall back to the H264 manifest.

Check out this Codepen with a real working example.

See the Pen Cloudinary - Multi-Codec Adaptive Bitrate Streaming by Robert Moseley (@mosedart) on CodePen.


At Cloudinary, we’re continuously conducting research on ABS and will make ongoing enhancements to make it easier to execute on multiple codecs. Try it out for yourself and tell us about your experience in the Comments section below. Also, feel free to check out our HTML5 Video Player

Recent Blog Posts

How to Customize Cloudinary's eCommerce Android App

Recently we added the Cloudinary Demo - eCommerce App to the Google Play Store. This app demonstrates the best practices for optimal delivery of images on a storefront, including category pages, product pages, and a shopping cart. At the time, we published Introducing the Cloudinary Demo Android App, Part 1, which provided an under-the-hood tour of how the eCommerce Android App was designed and how Cloudinary was integrated throughout.

Read more
The New Cloudinary Flexible and Dynamic Website Design

Last month we unveiled a refresh of our website. As with all redesigns, there were numerous drivers behind the rollout. First and foremost, we’re a company that understands the power of visual storytelling, and the ways in which a website must beautifully and clearly convey a brand’s story and offerings. Our website needed to better reflect our brand and our solutions. This post highlights some of the thinking behind our changes.

Read more
Auto-Generate Video Previews with Great Results Every Time

If you are an avid YouTube viewer or a Netflix binger, you might have noticed some fascinating changes in their latest preview thumbnails. They sure are attention grabbing, aren’t they?

Video previews are becoming more and more popular, not the least because they entice viewers, attract clickthroughs and, more often than not, gain a happy and loyal audience. Problem is, the task sounds like an art in itself. Just how can you go about creating appealing video previews without having to rely on movie professionals? Have you ever wished that you could quickly create video previews like those on YouTube and Netflix, but still rest assured that they'll include the good stuff?

Read more