Cloudinary Blog

Video Optimization, Part 2: Multi-Codec Adaptive Bitrate Streaming

Multi-Codec Adaptive Bitrate Streaming for Video Optimization

Part 1 of this series discusses the optimal way of delivering progressive video streams by taking advantage of modern, efficient codecs. That approach is ideal for short-form (under 60 seconds) videos and for videos displayed at a low resolution, such as ads and previews. But what if you're delivering longer videos with a higher resolution? No matter that you could offer them as a single file through progressive streaming, your viewers might still run into issues, such as buffering, slow start of playbacks, or even playback failures.

As sharp as modern mobile-device screens have become, they can't display all the pixels in high-resolution videos (above 1,080p). Hence, delivering a video too large for the screen is not only a waste of resources and increased bandwidth costs on your part, but also a drain on your audience’s bandwidth, lengthening the processing time on devices, consuming more battery, and rendering buffering a likely occurrence.

An effective solution is Adaptive Bitrate Streaming (ABR). Read on to find out how to combine modern codecs with ABR to deliver, through automation with Cloudinary, a lightweight and optimal experience to viewers.


Sign up for Cloudinary free today!


Adaptive Bitrate Streaming

Adaptive Bitrate Streaming

While on YouTube, Netflix, Hulu, or a similar service, if you notice the picture quality changing from good to pixelated or vice versa, that’s ABR. Behind the scenes, instead of buffering your video when the network is slow, ABR switches dynamically to a lower-bitrate (lower-quality) version of the video that can be handled by the connection speed.

ABR also takes into account your device’s resolution. So, if you're watching a video on an iPhone X, ABR does not deliver the video’s 4K version—even if your bandwidth can accommodate such a large file—because not enough pixels are available in the display for that higher-resolution version. The magic here is that all that happens automatically with Cloudinary; you just need to configure the Cloudinary Video Player with the relevant options.

For a more in-depth discussion of ABR, see the post Plug-and-Play Adaptive Bitrate Streaming With HLS and MPEG-DASH.

Definition of Bitrate

Before delving into how to deliver an optimal adaptive experience, let's talk about the main prerequisite. Bitrate refers to the number of bits it takes to encode a one-second video. Thus, a 10-second video with a 10-MB file size has an average bitrate of 8 mbps (1 megabyte per second * 8 bits per byte). To change the video bitrate, first note this formula:

Bitrate = total pixels/second _bits/pixels * or total pixels/frame x frames/second * bits/total pixels

Hence, for a lower bitrate, you can reduce the total number of pixels by doing one of the following:

  • Reduce the resolution of the video.
  • Lower the frame rate.
  • Compress the bits per pixel with a more advanced video codec—akin to what you would do for an image.

Codecs

Part 1 elaborates on modern codecs, such as HEVC (H.265) and VP9, which render sharing pixels across frames more efficient, encoding the key frames while maintaining the visual fidelity of videos. In fact, they are about 30-50 percent more efficient than the H.264 standard. Currently, however, HEVC and VP9 do not enjoy universal browser or device support.

Not to worry. As described in Part 1, you can present your video player with multiple codec options in the adaptive bitrate manifest, which would then choose the first codec it knows how to decode. In fact, about 85 percent of Internet traffic uses one of the modern codecs, meaning that your total bandwidth would be reduced by 25 to 43 percent. You can easily calculate how cost-effective this approach would be for you.

Creation of Streaming Profiles for Your Video Files

To set up codec options, first create the streaming profiles to build the manifests. Think of a streaming profile as a list of all the variations you want to create for ABR of your videos. You can specify in a profile the aspect ratio, the resolutions, the bitrates, and any additional Cloudinary transformations for your videos, such as watermarks, subtitles, other text, and splicing in other videos.

Create a maximum of three streaming profiles, one for each codec you want to support: VP9, H.265, and H.264. Before doing that, remember that your goal is to deliver the best quality with the available bitrate given the bandwidth constraint. Also note that—

  • Since H.265 and VP9 are more efficient, you can get the same picture quality for a lower bitrate. If saving bandwidth costs is your primary concern, this is probably the way to go.
  • At a given bandwidth availability, users whose devices support those modern codecs get a more superior visual experience.

Keep in mind that the content of your video could also affect your decision. For instance, for sports or action content, you might want to maintain a higher bitrate at lower resolutions.

Creating Bitrate Profiles

Before creating the profiles, see the Cloudinary API code for creating a streaming profile for H.264: Creating Custom Streaming Profiles

Copy to clipboard
cloudinary.v2.api.create_streaming_profile('16x9_h264',
  {
    display_name: "16x9 H264 Codec",
    representations: [
    {transformation:{crop: "pad", width: 1920, aspect_ratio: "16:9", bit_rate: "8500k"}},
    {transformation:{crop: "pad", width: 1280, aspect_ratio: "16:9", bit_rate: "5500k"}},
    {transformation:{crop: "pad", width: 960, aspect_ratio: "16:9", bit_rate: "3500k"}},
    {transformation:{crop: "pad", width: 640, aspect_ratio: "16:9", bit_rate: "2000k"}},
    {transformation:{crop: "pad", width: 480, aspect_ratio: "16:9", bit_rate: "800k"}}
  ] },
  function(error, result){console.log(result);});

Copy that code, paste it into another version, and make the necessary edits for H.265, as follows:

Copy to clipboard
cloudinary.v2.api.create_streaming_profile('16x9_h265',
  {
    display_name: "16x9 H265 Codec",
    representations: [
    {transformation:{crop: "pad", video_codec: "h265", width: 1920, aspect_ratio: "16:9", bit_rate: "5500k"}},
    {transformation:{crop: "pad", video_codec: "h265", width: 1280, aspect_ratio: "16:9", bit_rate: "3500k"}},
    {transformation:{crop: "pad", video_codec: "h265", width: 960, aspect_ratio: "16:9", bit_rate: "2000k"}},
    {transformation:{crop: "pad", video_codec: "h265", width: 640, aspect_ratio: "16:9", bit_rate: "800k"}},
    {transformation:{crop: "pad", video_codec: "h265", width: 480, aspect_ratio: "16:9", bit_rate: "500k"}}
  ] },
  function(error, result){console.log(result);});

Notice the lower bitrates on H.265.

Now create a profile for VP9 by using the same bitrates you did for H.265:

Copy to clipboard
  cloudinary.v2.api.create_streaming_profile('16x9_vp9',
  {
    display_name: "16x9 vp9 Codec",
    representations: [
    {transformation:{crop: "pad", video_codec: "vp9", width: 1920, aspect_ratio: "16:9", bit_rate: "5500k"}},
    {transformation:{crop: "pad", video_codec: "vp9", width: 1280, aspect_ratio: "16:9", bit_rate: "3500k"}},
    {transformation:{crop: "pad", video_codec: "vp9", width: 960, aspect_ratio: "16:9", bit_rate: "2000k"}},
    {transformation:{crop: "pad", video_codec: "vp9", width: 640, aspect_ratio: "16:9", bit_rate: "800k"}},
    {transformation:{crop: "pad", video_codec: "vp9", width: 480, aspect_ratio: "16:9", bit_rate: "500k"}}
  ] },
  function(error, result){console.log(result);});

The above bitrates are examples only. Though they should work fine in most cases, be sure to test them out and tweak the bitrates to meet your own standards. Keep in mind that a given bitrate at a constant resolution would result in a video of higher visual quality in VP9 and H.265 than in H.264.

Now create all those versions upon upload of a new video, as follows:

CAUTION: This approach results in numerous transformations—something well worth considering because video encoding is expensive. You can offset part or even all of that cost by reducing the bandwidth, depending on the ratio of bandwidth consumption to the number of videos to be encoded. All in all, it’s cost effective for situations in which the bandwidth savings are higher than the processing cost you incur. As stated earlier, the above approach yields a 25- to 43-percent reduction in total outgoing video bandwidth. With Cloudinary, 1 TB of outgoing bandwidth equals the cost of one million transformations. You can easily calculate the total cost or cost savings.

Uploading Videos

Next, upload a new video and create the various versions. The above example would read like this:

Copy to clipboard
cloudinary.v2.uploader.upload('video-file.mp4', {resource_type: 'video', 
    eager: [{format: 'm3u8', streaming_profile: '16x9_h264'},
      {format: 'm3u8', streaming_profile: '16x9_h265'},
      {format: 'mpd', streaming_profile: '16x9_vp9'}
      ],                                   
    eager_async: true,
    eager_notification_url: `https://`mysite`/notify_endpoint', public_id:`yourPublicId`}, function(result) {console.log(result); });

Replace all the variables in bold font with the actual values. When your files are ready, you’ll get an async response to the endpoint under eager_notification_url. Remember that you’re creating a lot of files here and, even though VP9 and H.265 codecs are extremely efficient consumers of bandwidth, they take three to five times as long to encode than H.264. However, with parallel processing, the operation would take the same amount of time despite a higher CPU requirement.

When the upload is complete, the following three manifests are displayed, which you’ll need for the next step:

Copy to clipboard
https://res.cloudinary.com/demo-robert/video/upload/sp_16x9_h264/yourPublicId.m3u8
https://res.cloudinary.com/demo-robert/video/upload/sp_16x9_h265/yourPublicId.m3u8
https://res.cloudinary.com/demo-robert/video/upload/sp_16x9_vp9/yourPublicId.mpd

Now decide which manifest to pick for each visitor. (As reference, see this multi-codec HLS/DASH CodePen). Check for support for H.265 and for VP9. If neither works, fall back to the H.264 manifest.

This CodePen contains a real working example.

See the Pen Cloudinary and Multi-Codec Adaptive Bitrate Streaming by Robert Moseley (@mosedart) on CodePen.

Conclusion

At Cloudinary, we’re continuously conducting research on ABR and developing enhancements to make it easier to run codecs. Try out the process for yourself and tell us about your experience in the Comments section below. Also, do check out our HTML5 Video Player.


Further Reading on Video Transformations

Recent Blog Posts

Partner news: Cloudinary-Getty Images Integration

Supported by intelligent automation, Cloudinary serves as an effective conduit between media asset management and delivery so you can take maximum advantage of assets, compress workflows, and build and coordinate engaging and inspiring customer experiences. Through Cloudinary’s Digital Asset Management (DAM) solution, which employs the company’s innovative image and video APIs, creative and marketing teams can benefit from them, as well as from many AI-powered and automated capabilities. As a result, you can transform, optimize, and deliver media at scale on an intuitive UI.

Read more
Why Audio in Video Matters

Many content creators and consumers tend to regard video as visuals, but that’s only part of the experience. Immersive video content includes strong audio. Just like in a movie, the audio for video content comprises many components: the narrator or subjects, the background music that sets the mood and draws viewers in, sound effects, and so forth.

Read more

For Developers: the HTML <picture> Element Explained

By Amarachi Amaechi
For Developers: the HTML <picture> Element Explained

We all know the good ol', tireless <img> element, which has been a long-time go-to for inserting graphics into webpages. Time doesn’t stop, however, and neither do technological advancements. So, let’s get you up to speed with the element’s modern alternative: the <picture> element.

Read more