Cloudinary Blog

Plug-and-play adaptive bitrate streaming with HLS and MPEG-DASH

HTTP Live Streaming Without the Hard Work | Cloudinary Blog

You know that moment when you click a video link on your phone while on your way to grab a coffee from the office kitchen, but then you get that annoying buffering icon, and just give up? That video might have been interesting, maybe even valuable, but it’s just not worth your time.

Millions of users repeat this experience every day: stealing a couple minutes before starting work, stretching the time before going to sleep or getting out of bed, grabbing a quick break in between other planned tasks, etc. In any of these scenarios, a short video can be a great way to learn more about a concept, product, or get the latest news, but our ‘free’ moments are precious and we are not going to waste them waiting for heavy videos to download to our phones and other devices. Even when we do have more time to spare, we find ourselves too impatient to wait for any video that buffers for too long.

If you deliver video on your website, you understand that great video content is of little value if it isn't delivered quickly enough to get your users watching and then keep their attention until the end. You may also know that the best way to minimize these incredibly costly failures is to deliver video using adaptive bitrate streaming.

Adaptive bitrate streaming is a video delivery technique that adjusts the quality of a video stream in real time according to detected bandwidth and CPU capacity of each user.

Two of the most popular adaptive bitrate streaming formats are HLS (HTTP Live Streaming) and MPEG-DASH (Dynamic Adaptive Streaming over HTTP). If you can deliver video in both of these formats, you can probably minimize buffering time for just about all of your users. But each of these formats requires outputting every video multiple times at different qualities along with a set of additional required files. This process is generally both complex and tedious. Services that help you create these files can be costly, and most don’t provide an end-to-end solution.

You can simplify the creation of these files using Cloudinary’s adaptive streaming profile functionality, which enables you to automatically generate all of the required video files for each of the desired quality and bitrate levels, along with the corresponding index and master files. You just upload a video in any format, plug in the name of one of Cloudinary's predefined streaming profiles in an eager transformation request (or specify a custom streaming profile that you defined with the Admin API), and then deliver the automatically generated, fully packaged .m3u8 or .mpd file using the HTTP Live Streaming or MPEG-DASH player of your choice. Literally Plug... and Play!

How adaptive bitrate streaming works

Providing a video using adaptive bitrate streaming means that you actually provide several versions (known as representations or variants) of your video, each with different qualities, bitrates, and codecs. Each video file must also be accompanied by an index file that specifies predefined segments or chunks of the video. These segments are usually 2, 5, or 10 seconds long. Additionally, there is a master playlist that points to the available representations with additional information about each one.

adaptive bitmap streaming files, diagram

The adaptive streaming player uses this information to decide which of the available representations best matches a user’s network conditions or preferences at any time. It can switch to another representation with higher or lower quality at each segment if the network conditions change. This helps maintain continuous viewing with the best possible quality to maximize the user experience.

The example below simulates the process that an adaptive streaming player might follow as a user watches a video that’s delivered via adaptive streaming:

adaptive bitmap streaming simulation

 

User begins playing while connected to wifi. Video begins playing immediately using the smaller 640x360 representation and simultaneously buffers.

After a few seconds, the user expands to full screen, and since the buffering is sufficient, the client delivers the higher resolution 1280x720 representation.

 

The user manually selects the 1920x1080 representation from the player’s resolution selection box.

 

As the user walks from his home to his car and leaves his wifi range, he changes the player resolution selection back to automatic. The client recognizes the drop in network performance and reverts to 640x360 play.

HTTP Live Streaming and MPEG-DASH adaptive bitrate streaming formats

The two leading adaptive bitrate streaming formats are HLS (HTTP Live Streaming) and MPEG-DASH (Dynamic Adaptive Streaming over HTTP).

HLS is an adaptive streaming communications protocol created by Apple. The Safari browser can play HLS streams within a web page, iPhones, and iPod touch devices. Since v2, all Apple TV devices also include an HTTP Live Streaming client. Apple’s AppStore requires apps that stream videos longer than 10 minutes over a cellular network to deliver those videos via HLS adaptive streaming. For details, see https://developer.apple.com/library/ios/qa/qa1767/_index.html

MPEG-DASH is an international standard for streaming video, intended to be a standardized replacement for proprietary HTTP streaming technologies, but is currently not natively supported in iOS devices. In other browsers and devices, it requires a javascript library or a video player that supports MPEG-DASH.

As mentioned above, to deliver videos using adaptive bitrate streaming, you must generate multiple video representations, an index file per representation, and a master playlist. The formats and encoding for HLS and MPEG-DASH are different for each of these files. So, if you want to deliver both HTTP Live Streaming and MPEG-DASH formats, to cover both iOS and non-iOS devices, then you need to double the effort for every video you want to deliver. Additionally, for MPEG-DASH, the best practice is to deliver the audio and video separately, so chalk up even more files and complexity.

Cloudinary streaming profiles

Generating so many video outputs and creating all the required supporting files for every video you want to deliver, is a huge and potentially expensive challenge.

To address this challenge, Cloudinary enables you to pick from a set of predefined streaming profiles or create your own custom profile. A Cloudinary streaming profile comprises a set of video representation definitions with various qualities, bitrates, and codecs. For example, the 4k predefined profile specifies 8 different representations in 16*9 aspect ratio, ranging from extremely high quality to audio-only.

Once you pick (or define) a streaming profile, you simply upload your video file with a single eager transformation that instructs Cloudinary to generate all the required files for the requested profile in either HTTP Live Streaming or MPEG-DASH format. If you want to deliver both formats, all it takes is two eager transformations within your upload command, instead of one.

For example, this Ruby code eagerly generates the big_buck_bunny video in both HTTP Live Streaming and MPEG-DASH formats using the full_hd built-in streaming profile, which yields 6 representations of the movie at a variety of quality levels.

Cloudinary::Uploader.upload("big_buck_bunny.mp4", :resource_type => :video, 
  :eager => [
     {:streaming_profile => "full_hd", :format => "m3u8"},
     {:streaming_profile => "full_hd", :format => "mpd"}], 
  :eager_async => true,
  :eager_notification_url => "http://mysite/notify_endpoint",
  :public_id => "bb_bunny")

There are seven different predefined adaptive streaming profiles to choose from. The same profiles are relevant for both HLS and DASH. For full details on the available profiles, see Predefined streaming profiles in the Cloudinary documentation. You can use the Admin API to fine-tune the predefined profiles to fit your needs, or to create your own custom streaming profiles.

When the eager upload is complete, the only thing left for you to do is to embed the relevant adaptive streaming client player(s) in your application. There are a number of open source and closed source HLS and DASH client players available.

While you play an m3u8 or mpd file generated using an adaptive streaming profile, you can take a look at the way the client player selects from your representations using the browser tools, such as the Chrome DevTools console:

browser network console showing the active video representation

For example, you can see above that the highlighted segment played using the 480x360 resolution with 800k bitrate, while the following segment played the representation with 640x480 resolution and 2m bitrate.

The higher-level the streaming profile you select, the more levels the client player has to choose from, to serve up the best possible quality that will enable continuous playback for each user’s current network conditions. However, do keep in mind that higher level profiles will use more storage and transformation resources in your Cloudinary account. If this is an issue, you may want to take advantage of the ‘lean’ predefined profiles, which include fewer representations, with somewhat bigger jumps between lower and higher quality. Also, remember that even though you now only need a single line of code to prepare your HLS and/or MPEG-DASH output, these transformations involve a lot of overhead and must be done eagerly.

(Note: Technically, Cloudinary does support transforming and delivering these files on-the-fly in the case that the total size of all the files together is less than 100MB, but due to the long encoding time, this on-the-fly option should be used only for debugging purposes.)

You can also apply other transformations alongside your streaming profile transformation. For example, the following URL plays the bb_bunny video in HTTP Live Streaming format with a small logo overlaid in the top left corner of the movie.

(Note: The above URL can be viewed only in devices or players that support HLS)

Applying transformations in this manner applies the same transformation to all representations. You can also apply different transformations to different representations in a profile, for example, to include a text overlay showing the user which resolution is currently playing. To do this, you need to create a custom streaming profile.

Custom Streaming Profiles

The predefined adaptive streaming profiles that Cloudinary provides are based on a set of selected best practices and one of them is likely to be a good fit for most standard scenarios.

But what if none of the provided profiles exactly answer your needs? For example, you might want a different number of representations or different divisions of quality available for your profile. Or you might want to apply special transformations for different representations within the profile. To create a customized streaming profile, you use the streaming_profiles method of the Admin API.

For example, below we've used the Ruby implementation of the Admin API to create an hd_debug profile, where each defined representation includes the relevant video transformations as well as text overlay that displays that resolution and bitrate. This helps us debug and watch how the client switches from one representation to another as the video buffer grows, as we resize the screen, or as network conditions change.

Cloudinary::Api.create_streaming_profile("hd_debug", :representations => 
[{:transformation =>
   [{:width=>320,:height=>240,:bit_rate=>'192k',:crop=>'limit'},
    {:overlay=>'text:Arial_24_bold:320x240--bitrate:192KB',
     :color=>'white',:background=>'rgb:00000020'},  
    {:border=>'6px_solid_rgb:00000020',:gravity=>'north',:y=>0.1,
     :flags=>'layer_apply'}]},
{:transformation =>
   [{:width=>480,:height=>270,:video_codec=>'h264:baseline:3.0',
     :bit_rate=>'800k',:crop=>'limit'},                
    {:overlay=>'text:Arial_24_bold:480x270--bitrate:800KB',
     :color=>'white',:background=>'rgb:00000020'}
    {:border=>'6px_solid_rgb:00000020',:gravity=>'north',:y=>0.1,
     :flags=>'layer_apply'}]},
{:transformation =>
   [{:width=>640,:height=>360,:video_codec=>'h264:baseline:3.0',
     :bit_rate=>'2m',:crop=>'limit'},
    {:overlay=>'text:Arial_24_bold:640x360--bitrate:2MB',
     :color=>'white',:background=>'rgb:00000020'},
    {:border=>'6px_solid_rgb:00000020',:gravity=>'north',:y=>0.1,
     :flags=>'layer_apply'}]},
{:transformation =>
   [{:width=>960,:height=>540,:video_codec=>'h264:main:3.1',
     :bit_rate=>'3500k',:crop=>'limit'},
    {:overlay=>'text:Arial_24_bold:960x540--bitrate:3.5MB',
     :color=>'white',:background=>'rgb:00000020'},      
    {:border=>'6px_solid_rgb:00000020',:gravity=>'north',:y=>0.1,
     :flags=>'layer_apply'}]},
{:transformation =>
   [{:width=>1280,:height=>720,:video_codec=>'h264:main:3.1',
     :bit_rate=>'5500k',:crop=>'limit'},
    {:overlay=>'text:Arial_24_bold:1280x720--bitrate:5.5MB',
     :color=>'white',:background=>'rgb:00000020'},
    {:border=>'6px_solid_rgb:00000020',:gravity=>'north',:y=>0.1,
     :flags=>'layer_apply'}]}])

After uploading the bb_bunny video with an eager transformation requesting to use this hd_debug profile in HLS (.m3u8) format, you can play the HLS video below natively in your mobile device or in an iOS browser, such as Safari. Give it a try!

https://res.cloudinary.com/cloudinary/video/upload/sp_hd_debug/bb_bunny.m3u8

debug profile

(If you don't have a browser that supports HLS, navigate to hlsplayer.net and paste in the link above.)

Just Imagine...

Remember that "moment when" moment I described at the start of this post? You’ve been there. I’ve been there. But just imagine that with a couple lines of code to eagerly generate all your adaptive streaming files, your website users won’t have to 'be there' anymore.

To learn more, see Adaptive bitrate streaming - HLS and MPEG-DASH and the streaming_profiles method of the Admin API in the Cloudinary documentation .

Adaptive bitrate streaming profiles for HLS and MPEG-DASH are supported for all Cloudinary plans, including the Free Plan. Go ahead, try it out for yourself.

Recent Blog Posts

An Eye-Opening Talk: Building Apps for the Next Billion Users in Africa

William (iChuloo) Imoh, who hails from Lagos, Nigeria, recently embarked on a U.S. speaking tour, February 20-March 12, during which he powwowed with technical and product teams and communities at such renowned enterprises as Netlify, Pluralsight, Lucidchart, Twilio, and more in Salt Lake City, Dallas, Las Vegas, and San Francisco. On March 5, he gave an enlightening talk, entitled International Developers and Development: Building for the Next Billion Users at Cloudinary in Santa Clara, California. Below is a synopsis. For details, see the related slides.

Read more
The Debut of the Cloudinary Customer Advisory Board

Focus on customers has always been Cloudinary’s mantra. Because we owe them our success, we are constantly reaching out to our customers, not just for feedback on our offerings, but also for their vision, wish list, and buy-in of what Cloudinary can do to meet their needs and make them succeed. About six months ago, it occurred to us that it would be beneficial if we could meet regularly with those who are behind innovation at our key customers—executives, product gurus, developers, content managers—to swap strategies, product roadmaps, best practices, and such. In particular, we’d like to solicit actionable feedback as a foundation for our plans of product enhancements.

Read more
Media Management With the Cloudinary-Netlify CMS Integration

Static sites and the JAMstack are quickly becoming a standard for developing safe and performant websites with an optimal workflow for developers. Netlify CMS (not to be confused with the company that created it, Netlify) is an open source content management solution that works especially with static site generators such as Gatsby, Hugo, etc... enabling content storage in your Git repository along with your code for easier versioning, multichannel publishing, and direct content updates in Git.

Read more
Vitaly Friedman's Insights on Media Conferences

Vitaly Friedman is a die-hard devotee of beautiful content. Born in Minsk, Belarus, he studied Computer Science and Mathematics in Germany, unearthing in himself a passion for typography, writing, and design in the interim. After a six-year stint as a freelance designer and developer, he co-founded Smashing Magazine, a leading online publication on web design and development. You can follow SmashingMag on Twitter @SmashingMag.

Read more