Cloudinary Blog

HTTP Live Streaming Without the Hard Work

HTTP Live Streaming Without the Hard Work

Plug and play image for HTTP Live Streaming and MPEG-DASH

You know that moment when you click a video link on your phone while on your way to grab a coffee from the office kitchen, but then you get that annoying buffering icon, and just give up? That video might have been interesting, maybe even valuable, but it’s just not worth your time.

Millions of users repeat this experience every day: stealing a couple minutes before starting work, stretching the time before going to sleep or getting out of bed, grabbing a quick break in between other planned tasks, etc. In any of these scenarios, a short video can be a great way to learn more about a concept, product, or get the latest news, but our ‘free’ moments are precious and we are not going to waste them waiting for heavy videos to download to our phones and other devices. Even when we do have more time to spare, we find ourselves too impatient to wait for any video that buffers for too long.

If you deliver video on your website, you understand that great video content is of little value if it isn't delivered quickly enough to get your users watching and then keep their attention until the end. You may also know that the best way to minimize these incredibly costly failures is to deliver video using adaptive bitrate streaming.

Adaptive bitrate streaming is a video delivery technique that adjusts the quality of a video stream in real time according to detected bandwidth and CPU capacity of each user.

Two of the most popular adaptive bitrate streaming formats are HLS (HTTP Live Streaming) and MPEG-DASH (Dynamic Adaptive Streaming over HTTP). If you can deliver video in both of these formats, you can probably minimize buffering time for just about all of your users. But each of these formats requires outputting every video multiple times at different qualities along with a set of additional required files. This process is generally both complex and tedious. Services that help you create these files can be costly, and most don’t provide an end-to-end solution.

You can simplify the creation of these files using Cloudinary’s adaptive streaming profile functionality, which enables you to automatically generate all of the required video files for each of the desired quality and bitrate levels, along with the corresponding index and master files. You just upload a video in any format, plug in the name of one of Cloudinary's predefined streaming profiles in an eager transformation request (or specify a custom streaming profile that you defined with the Admin API), and then deliver the automatically generated, fully packaged .m3u8 or .mpd file using the HTTP Live Streaming or MPEG-DASH player of your choice. Literally Plug... and Play!

How adaptive bitrate streaming works

Providing a video using adaptive bitrate streaming means that you actually provide several versions (known as representations or variants) of your video, each with different qualities, bitrates, and codecs. Each video file must also be accompanied by an index file that specifies predefined segments or chunks of the video. These segments are usually 2, 5, or 10 seconds long. Additionally, there is a master playlist that points to the available representations with additional information about each one.

adaptive bitmap streaming files, diagram

The adaptive streaming player uses this information to decide which of the available representations best matches a user’s network conditions or preferences at any time. It can switch to another representation with higher or lower quality at each segment if the network conditions change. This helps maintain continuous viewing with the best possible quality to maximize the user experience.

The example below simulates the process that an adaptive streaming player might follow as a user watches a video that’s delivered via adaptive streaming:

adaptive bitmap streaming simulation

 

User begins playing while connected to wifi. Video begins playing immediately using the smaller 640x360 representation and simultaneously buffers.

After a few seconds, the user expands to full screen, and since the buffering is sufficient, the client delivers the higher resolution 1280x720 representation.

 

The user manually selects the 1920x1080 representation from the player’s resolution selection box.

 

As the user walks from his home to his car and leaves his wifi range, he changes the player resolution selection back to automatic. The client recognizes the drop in network performance and reverts to 640x360 play.

HTTP Live Streaming and MPEG-DASH adaptive bitrate streaming formats

The two leading adaptive bitrate streaming formats are HLS (HTTP Live Streaming) and MPEG-DASH (Dynamic Adaptive Streaming over HTTP).

HLS is an adaptive streaming communications protocol created by Apple. The Safari browser can play HLS streams within a web page, iPhones, and iPod touch devices. Since v2, all Apple TV devices also include an HTTP Live Streaming client. Apple’s AppStore requires apps that stream videos longer than 10 minutes over a cellular network to deliver those videos via HLS adaptive streaming. For details, see https://developer.apple.com/library/ios/qa/qa1767/_index.html

MPEG-DASH is an international standard for streaming video, intended to be a standardized replacement for proprietary HTTP streaming technologies, but is currently not natively supported in iOS devices. In other browsers and devices, it requires a javascript library or a video player that supports MPEG-DASH.

As mentioned above, to deliver videos using adaptive bitrate streaming, you must generate multiple video representations, an index file per representation, and a master playlist. The formats and encoding for HLS and MPEG-DASH are different for each of these files. So, if you want to deliver both HTTP Live Streaming and MPEG-DASH formats, to cover both iOS and non-iOS devices, then you need to double the effort for every video you want to deliver. Additionally, for MPEG-DASH, the best practice is to deliver the audio and video separately, so chalk up even more files and complexity.

Cloudinary streaming profiles

Generating so many video outputs and creating all the required supporting files for every video you want to deliver, is a huge and potentially expensive challenge.

To address this challenge, Cloudinary enables you to pick from a set of predefined streaming profiles or create your own custom profile. A Cloudinary streaming profile comprises a set of video representation definitions with various qualities, bitrates, and codecs. For example, the 4k predefined profile specifies 8 different representations in 16*9 aspect ratio, ranging from extremely high quality to audio-only.

Once you pick (or define) a streaming profile, you simply upload your video file with a single eager transformation that instructs Cloudinary to generate all the required files for the requested profile in either HTTP Live Streaming or MPEG-DASH format. If you want to deliver both formats, all it takes is two eager transformations within your upload command, instead of one.

For example, this Ruby code eagerly generates the big_buck_bunny video in both HTTP Live Streaming and MPEG-DASH formats using the full_hd built-in streaming profile, which yields 6 representations of the movie at a variety of quality levels.

Cloudinary::Uploader.upload("big_buck_bunny.mp4", :resource_type => :video, 
  :eager => [
     {:streaming_profile => "full_hd", :format => "m3u8"},
     {:streaming_profile => "full_hd", :format => "mpd"}], 
  :eager_async => true,
  :eager_notification_url => "http://mysite/notify_endpoint",
  :public_id => "bb_bunny")

There are seven different predefined adaptive streaming profiles to choose from. The same profiles are relevant for both HLS and DASH. For full details on the available profiles, see Predefined streaming profiles in the Cloudinary documentation. You can use the Admin API to fine-tune the predefined profiles to fit your needs, or to create your own custom streaming profiles.

When the eager upload is complete, the only thing left for you to do is to embed the relevant adaptive streaming client player(s) in your application. There are a number of open source and closed source HLS and DASH client players available.

While you play an m3u8 or mpd file generated using an adaptive streaming profile, you can take a look at the way the client player selects from your representations using the browser tools, such as the Chrome DevTools console:

browser network console showing the active video representation

For example, you can see above that the highlighted segment played using the 480x360 resolution with 800k bitrate, while the following segment played the representation with 640x480 resolution and 2m bitrate.

The higher-level the streaming profile you select, the more levels the client player has to choose from, to serve up the best possible quality that will enable continuous playback for each user’s current network conditions. However, do keep in mind that higher level profiles will use more storage and transformation resources in your Cloudinary account. If this is an issue, you may want to take advantage of the ‘lean’ predefined profiles, which include fewer representations, with somewhat bigger jumps between lower and higher quality. Also, remember that even though you now only need a single line of code to prepare your HLS and/or MPEG-DASH output, these transformations involve a lot of overhead and must be done eagerly.

(Note: Technically, Cloudinary does support transforming and delivering these files on-the-fly in the case that the total size of all the files together is less than 100MB, but due to the long encoding time, this on-the-fly option should be used only for debugging purposes.)

You can also apply other transformations alongside your streaming profile transformation. For example, the following URL plays the bb_bunny video in HTTP Live Streaming format with a small logo overlaid in the top left corner of the movie.

(Note: The above URL can be viewed only in devices or players that support HLS)

Applying transformations in this manner applies the same transformation to all representations. You can also apply different transformations to different representations in a profile, for example, to include a text overlay showing the user which resolution is currently playing. To do this, you need to create a custom streaming profile.

Custom Streaming Profiles

The predefined adaptive streaming profiles that Cloudinary provides are based on a set of selected best practices and one of them is likely to be a good fit for most standard scenarios.

But what if none of the provided profiles exactly answer your needs? For example, you might want a different number of representations or different divisions of quality available for your profile. Or you might want to apply special transformations for different representations within the profile. To create a customized streaming profile, you use the streaming_profiles method of the Admin API.

For example, below we've used the Ruby implementation of the Admin API to create an hd_debug profile, where each defined representation includes the relevant video transformations as well as text overlay that displays that resolution and bitrate. This helps us debug and watch how the client switches from one representation to another as the video buffer grows, as we resize the screen, or as network conditions change.

Cloudinary::Api.create_streaming_profile("hd_debug", :representations => 
[{:transformation =>
   [{:width=>320,:height=>240,:bit_rate=>'192k',:crop=>'limit'},
    {:overlay=>'text:Arial_24_bold:320x240--bitrate:192KB',
     :color=>'white',:background=>'rgb:00000020'},  
    {:border=>'6px_solid_rgb:00000020',:gravity=>'north',:y=>0.1,
     :flags=>'layer_apply'}]},
{:transformation =>
   [{:width=>480,:height=>270,:video_codec=>'h264:baseline:3.0',
     :bit_rate=>'800k',:crop=>'limit'},                
    {:overlay=>'text:Arial_24_bold:480x270--bitrate:800KB',
     :color=>'white',:background=>'rgb:00000020'}
    {:border=>'6px_solid_rgb:00000020',:gravity=>'north',:y=>0.1,
     :flags=>'layer_apply'}]},
{:transformation =>
   [{:width=>640,:height=>360,:video_codec=>'h264:baseline:3.0',
     :bit_rate=>'2m',:crop=>'limit'},
    {:overlay=>'text:Arial_24_bold:640x360--bitrate:2MB',
     :color=>'white',:background=>'rgb:00000020'},
    {:border=>'6px_solid_rgb:00000020',:gravity=>'north',:y=>0.1,
     :flags=>'layer_apply'}]},
{:transformation =>
   [{:width=>960,:height=>540,:video_codec=>'h264:main:3.1',
     :bit_rate=>'3500k',:crop=>'limit'},
    {:overlay=>'text:Arial_24_bold:960x540--bitrate:3.5MB',
     :color=>'white',:background=>'rgb:00000020'},      
    {:border=>'6px_solid_rgb:00000020',:gravity=>'north',:y=>0.1,
     :flags=>'layer_apply'}]},
{:transformation =>
   [{:width=>1280,:height=>720,:video_codec=>'h264:main:3.1',
     :bit_rate=>'5500k',:crop=>'limit'},
    {:overlay=>'text:Arial_24_bold:1280x720--bitrate:5.5MB',
     :color=>'white',:background=>'rgb:00000020'},
    {:border=>'6px_solid_rgb:00000020',:gravity=>'north',:y=>0.1,
     :flags=>'layer_apply'}]}])

After uploading the bb_bunny video with an eager transformation requesting to use this hd_debug profile in HLS (.m3u8) format, you can play the HLS video below natively in your mobile device or in an iOS browser, such as Safari. Give it a try!

https://res.cloudinary.com/cloudinary/video/upload/sp_hd_debug/bb_bunny.m3u8

debug profile

(If you don't have a browser that supports HLS, navigate to hlsplayer.net and paste in the link above.)

Just Imagine...

Remember that "moment when" moment I described at the start of this post? You’ve been there. I’ve been there. But just imagine that with a couple lines of code to eagerly generate all your adaptive streaming files, your website users won’t have to 'be there' anymore.

To learn more, see Adaptive bitrate streaming - HLS and MPEG-DASH and the streaming_profiles method of the Admin API in the Cloudinary documentation .

Adaptive bitrate streaming profiles for HLS and MPEG-DASH are supported for all Cloudinary plans, including the Free Plan. Go ahead, try it out for yourself.

Recent Blog Posts

Video Manipulations and Delivery for Angular Video Apps

On social media, videos posted by users constitute a significant amount of the content appeal on those platforms. From upload to manipulation to delivery, a smooth, efficient, and effective pipeline for the posting process is mandatory to ensure consistent user sessions and their steadily increasing volume. However, building such an infrastructure is a complex, labor-intensive, and problem-prone undertaking.

Read more
Green Screen Queen: Dynamic Video Transparency Fit For Royalty

If you were reading your social media or news feeds on or around June 11 this year, no doubt you came across your fair share of posts about Queen Elizabeth and her outfit-color faux pas. For her 90th birthday, she chose a solid neon green suit, and it didn't take long for Photoshop fanatics to suggest alternative designs for the Queen's green-screen threads.

Read more
Content-Aware Automatic Cropping for Video

Delivering videos according to the aspect ratios defined by social media for multiple devices and platforms is a growing challenge. The continuously rising volume of vertical videos and the corresponding increase in video traffic on mobile devices (now up to 57% of online videos watched) have only exacerbated the situation, with no letup in sight.

Read more
Use a custom function in the image delivery pipeline

Cloudinary offers a wide array of image manipulations and effects to apply to images as part of our image-processing pipeline, helping to ensure that your images fit the graphic design of your website or mobile application. Cloudinary is an open platform, and you can use our APIs, Widgets and UI to build the media management flow that matches your needs.

Read more
OpenText™ TeamSite – Cloudinary Plugin

Tired of depending on other teams or software to create assets in multiple sizes for your responsive web site?

Does importing asset files into TeamSite slow down your web content publishing?

Klish Group is pleased to introduce the OpenText TM TeamSite – Cloudinary connector. Customers of the OpenText TM TeamSite web content management platform can now browse and select images in the same way they always have. Authors can just browse and select the image they want to use and Cloudinary will automatically deliver it in the optimal format and quality to the customer requesting it.

Read more