Cloudinary Blog

Muted Videos and Subtitles

Muted Videos and Subtitles

The bane of our existence is the lack of efficient ways for tackling the plethora of recurring tasks in our lives. One of those tasks is surfing the internet. We consume a lot of web content daily, of which a large percentage are images and videos. We’re constantly quickly scrolling through 30-second videos or checking out pictures of cute items we’d like to buy in our free time.

Some of those videos are in our native language; others are in foreign languages that we don’t understand. Sometimes we still enjoy the latter, however, because of the video’s graphical appeal. Other times, we might want to watch muted videos because of a loud immediate environment or an absence of headsets.

The Value of Subtitles

Subtitles are a superb feature for videos, muted or not.

Have you ever downloaded or watched someone download an SRT (.srt) file just to have the subtitles show up while watching a movie? As a software developer who builds apps that involve videos, you must inculcate the capacity that enables users to watch subtitles in videos.

Addition of Subtitles in Videos

How can you add video subtitles? Just as you can manually download and add SRT files to videos for subtitles, you can programmatically add subtitles to videos with Cloudinary. In fact, you can upload multiple SRT or WebVTT files to Cloudinary as normal raw files.

Making the content of those files show up as subtitles on your videos that have already been uploaded to Cloudinary is as straightforward as overlaying the subtitles on them. See this demo:

Ruby:
Copy to clipboard
cl_video_tag("dog", :overlay=>{:resource_type=>"subtitles", :public_id=>"sample_sub_en.srt"})
PHP v1:
Copy to clipboard
cl_video_tag("dog", array("overlay"=>array("resource_type"=>"subtitles", "public_id"=>"sample_sub_en.srt")))
PHP v2:
Copy to clipboard
(new VideoTag('dog.mp4'))
  ->overlay(Overlay::source(Source::subtitles('sample_sub_en.srt')));
Python:
Copy to clipboard
CloudinaryVideo("dog").video(overlay={'resource_type': "subtitles", 'public_id': "sample_sub_en.srt"})
Node.js:
Copy to clipboard
cloudinary.video("dog", {overlay: {resource_type: "subtitles", public_id: "sample_sub_en.srt"}})
Java:
Copy to clipboard
cloudinary.url().transformation(new Transformation().overlay(new SubtitlesLayer().publicId("sample_sub_en.srt"))).videoTag("dog");
JS:
Copy to clipboard
cloudinary.videoTag('dog', {overlay: new cloudinary.SubtitlesLayer().publicId("sample_sub_en.srt")}).toHtml();
jQuery:
Copy to clipboard
$.cloudinary.video("dog", {overlay: new cloudinary.SubtitlesLayer().publicId("sample_sub_en.srt")})
React:
Copy to clipboard
<Video publicId="dog" >
  <Transformation overlay={{publicId: "sample_sub_en.srt", resourceType: "subtitles"}} />
</Video>
Vue.js:
Copy to clipboard
<cld-video publicId="dog" >
  <cld-transformation :overlay="{publicId: 'sample_sub_en.srt', resourceType: 'subtitles'}" />
</cld-video>
Angular:
Copy to clipboard
<cl-video public-id="dog" >
  <cl-transformation overlay="subtitles:sample_sub_en.srt">
  </cl-transformation>
</cl-video>
.NET:
Copy to clipboard
cloudinary.Api.UrlVideoUp.Transform(new Transformation().Overlay(new SubtitlesLayer().PublicId("sample_sub_en.srt"))).BuildVideoTag("dog")
Android:
Copy to clipboard
MediaManager.get().url().transformation(new Transformation().overlay(new SubtitlesLayer().publicId("sample_sub_en.srt"))).resourceType("video").generate("dog.mp4");
iOS:
Copy to clipboard
cloudinary.createUrl().setResourceType("video").setTransformation(CLDTransformation().setOverlay("subtitles:sample_sub_en.srt")).generate("dog.mp4")

Simply specify the overlay parameter l_subtitles, followed by the public ID of the raw file and its extension. That syntax works in the URL transformation as shown above.

Programmatically, you can code it this way:

Copy to clipboard
cloudinary.video("dog", {overlay: {resource_type: "subtitles", public_id: "sample_sub_en.srt"}})

Node.js

Furthermore, you can customize the way subtitles appear in the video: the color, appearance, font, etc. For example, here’s how to make subtitles yellow in color with a red background and a larger font:

Copy to clipboard
cloudinary.video("dog", {overlay: {font_family: "arial", font_size: 100, resource_type: "subtitles", public_id: "sample_sub_en.srt"}, color: "#ffff00", background: "red"})

Node.js

Generation of Transcripts and Subtitles for Videos

Another important step is to generate transcripts for your videos on the fly. If your platform contains a lot of user-generated video content, you cannot count on your users to also upload the SRT files for the videos they upload. Instead, your users depend on your platform to generate subtitles for their content. For instance, social-media users with numerous followers would expect that the app can and would generate subtitles for their uploaded videos and that, as soon as they turn on the subtitle feature, followers can immediately view the subtitles.

Cloudinary’s Google Video Transcript add-on enables you to automate the generation of speech-to-text transcripts of user-generated videos that have been uploaded to your site. For the best possible speech-recognition results, the add-on, which can translate videos in almost any language, applies powerful neural-network models to videos with Google's Cloud Speech API.

After a transcript becomes available, you can do the following:

  1. Automatically insert the transcript into your video in the form of subtitles.
  2. Parse the content of the transcript and display it on a webpage. That way, users can skim the content instead of watching the videos. This approach is extremely popular on learning platforms.

suptitle demo

Here’s the code for generating transcripts:

Copy to clipboard
cloudinary.v2.uploader.upload("learning_javascript.mp4", 
  { resource_type: "video", 
    raw_convert: "google_speech" },
  function(error, result) {console.log(result, error) });

Node.js

Once generation is complete, the add-on creates a new file with a .transcript extension in your Cloudinary account with the same public ID as your video file.

Note
If you include the notification_url parameter in your method call, the URL you specify receives a notification once the generation process concludes.

Depending on the size of the video file, the process might take a while.

You might wonder, “Why a .transcript file? Why not a .srt or .vtt file?” Well, you do have the option to have Cloudinary generate a .srt and a .vtt file. Just specify your preference, like this:

Copy to clipboard
cloudinary.v2.uploader.upload("learning_javascript.mp4", 
  { resource_type: "video", 
    raw_convert: "google_speech:srt:vtt" },
  function(error, result) {console.log(result, error) });

Node.js

When the process is complete, the add-on automatically uploads these three generated files to your account:

Copy to clipboard
.../raw/upload/learning_javascript.transcript
.../raw/upload/learning_javascript.srt
.../raw/upload/learning_javascript.vtt

Display of Generated Transcripts and Subtitles on Videos

The final step is to display the generated transcripts and subtitles on the video for delivery. It’s as simple as programmatically overlaying the files on the video, like this:

Copy to clipboard
cloudinary.video("learning_javascript", {overlay: {resource_type: "subtitles", public_id: "learning_javascript.transcript"}})

Node.js

The URL reads—

https://res.cloudinary.com/demo/video/upload/l_subtitles:learning_javascript.transcript/learning_javascript.mp4

References

Below are additional references:

Recent Blog Posts

Automate the Staging Process of Videos for Social Media

Rich and engaging media helps build customer engagement and trust but can be time consuming to stage. Developers save a tremendous amount of time by preparing videos for social media with Cloudinary. That’s because Cloudinary’s interface, widgets, and application programming interface (API) transform raw media into polished content, optimizing footage and enabling effortless customization and publishing.

Read more

Top Five Web-Video Formats of 2021

By William Imoh
The Five Most Popular Web-Video Formats and Streaming Protocols

Over the past 15 years, the video industry has undergone a significant change in video formats on the web. In particular, in the early 2010s, the 3GP format, which the 3rd Generation Partnership Project (3GPP) created for 3G-enabled mobile devices, went nearly extinct. The advancement of mobile devices and cellular networks has brought about the need for pioneers to build better formats for a faster user experience.

Read more
Cloudinary Introduces Integration With SAP Commerce Cloud

We’re excited to announce Cloudinary’s integration with SAP Commerce Cloud, through which the latter’s customers can significantly boost the visual media experience on their website or app.

SAP Commerce Cloud powers some of the largest e-commerce sites (B2C, B2B, and B2B2C businesses), complete with building blocks like storefront design and order management. Reinforced with Cloudinary’s laser-sharp focus on optimizing, managing, and delivering images and videos, the new extension will enable SAP Commerce Cloud customers to create unique and engaging visual experiences effortlessly.

Read more
Personalizing Video Email for Marketing Campaigns With Cloudinary

As critical as it is to engage with shoppers in order to succeed in e-commerce, old-style, boring emails are far from being effective. In fact, they tend to be annoying because no one likes to read formulaic, generic messages that are sent en masse. For better results, rethink your email marketing campaigns and try out creative strategies.

Read more

Building a Roommate-Matching App With Cloudinary and Jamstack

By Marcelo Ricardo de Oliveira
Building a Roommate-Matching App With Cloudinary and Jamstack

Roommate matching can be a pain—especially during the COVID pandemic when people don't want to meet in person. Matching apps like Flatmates, Roomster, and roommates.com are helpful, and if you're in the roommate-matching space, you know that great video is essential for those seeking roommates. Fortunately, Cloudinary can help.

Read more