Cloudinary Blog

Muted Videos and Subtitles

Muted Videos and Subtitles

The bane of our existence is the lack of efficient ways for tackling the plethora of recurring tasks in our lives. One of those tasks is surfing the internet. We consume a lot of web content daily, of which a large percentage are images and videos. We’re constantly quickly scrolling through 30-second videos or checking out pictures of cute items we’d like to buy in our free time.

Some of those videos are in our native language; others are in foreign languages that we don’t understand. Sometimes we still enjoy the latter, however, because of the video’s graphical appeal. Other times, we might want to watch muted videos because of a loud immediate environment or an absence of headsets.

The Value of Subtitles

Subtitles are a superb feature for videos, muted or not.

Have you ever downloaded or watched someone download an SRT (.srt) file just to have the subtitles show up while watching a movie? As a software developer who builds apps that involve videos, you must inculcate the capacity that enables users to watch subtitles in videos.

Addition of Subtitles in Videos

How can you add video subtitles? Just as you can manually download and add SRT files to videos for subtitles, you can programmatically add subtitles to videos with Cloudinary. In fact, you can upload multiple SRT or WebVTT files to Cloudinary as normal raw files.

Making the content of those files show up as subtitles on your videos that have already been uploaded to Cloudinary is as straightforward as overlaying the subtitles on them. See this demo:

Ruby:
Copy to clipboard
cl_video_tag("dog", :overlay=>{:resource_type=>"subtitles", :public_id=>"sample_sub_en.srt"})
PHP v1:
Copy to clipboard
cl_video_tag("dog", array("overlay"=>array("resource_type"=>"subtitles", "public_id"=>"sample_sub_en.srt")))
PHP v2:
Copy to clipboard
(new VideoTag('dog.mp4'))
  ->overlay(Overlay::source(Source::subtitles('sample_sub_en.srt')));
Python:
Copy to clipboard
CloudinaryVideo("dog").video(overlay={'resource_type': "subtitles", 'public_id': "sample_sub_en.srt"})
Node.js:
Copy to clipboard
cloudinary.video("dog", {overlay: {resource_type: "subtitles", public_id: "sample_sub_en.srt"}})
Java:
Copy to clipboard
cloudinary.url().transformation(new Transformation().overlay(new SubtitlesLayer().publicId("sample_sub_en.srt"))).videoTag("dog");
JS:
Copy to clipboard
cloudinary.videoTag('dog', {overlay: new cloudinary.SubtitlesLayer().publicId("sample_sub_en.srt")}).toHtml();
jQuery:
Copy to clipboard
$.cloudinary.video("dog", {overlay: new cloudinary.SubtitlesLayer().publicId("sample_sub_en.srt")})
React:
Copy to clipboard
<Video publicId="dog" >
  <Transformation overlay={{publicId: "sample_sub_en.srt", resourceType: "subtitles"}} />
</Video>
Vue.js:
Copy to clipboard
<cld-video publicId="dog" >
  <cld-transformation :overlay="{publicId: 'sample_sub_en.srt', resourceType: 'subtitles'}" />
</cld-video>
Angular:
Copy to clipboard
<cl-video public-id="dog" >
  <cl-transformation overlay="subtitles:sample_sub_en.srt">
  </cl-transformation>
</cl-video>
.NET:
Copy to clipboard
cloudinary.Api.UrlVideoUp.Transform(new Transformation().Overlay(new SubtitlesLayer().PublicId("sample_sub_en.srt"))).BuildVideoTag("dog")
Android:
Copy to clipboard
MediaManager.get().url().transformation(new Transformation().overlay(new SubtitlesLayer().publicId("sample_sub_en.srt"))).resourceType("video").generate("dog.mp4");
iOS:
Copy to clipboard
cloudinary.createUrl().setResourceType("video").setTransformation(CLDTransformation().setOverlay("subtitles:sample_sub_en.srt")).generate("dog.mp4")

Simply specify the overlay parameter l_subtitles, followed by the public ID of the raw file and its extension. That syntax works in the URL transformation as shown above.

Programmatically, you can code it this way:

Copy to clipboard
cloudinary.video("dog", {overlay: {resource_type: "subtitles", public_id: "sample_sub_en.srt"}})

Node.js

Furthermore, you can customize the way subtitles appear in the video: the color, appearance, font, etc. For example, here’s how to make subtitles yellow in color with a red background and a larger font:

Copy to clipboard
cloudinary.video("dog", {overlay: {font_family: "arial", font_size: 100, resource_type: "subtitles", public_id: "sample_sub_en.srt"}, color: "#ffff00", background: "red"})

Node.js

Generation of Transcripts and Subtitles for Videos

Another important step is to generate transcripts for your videos on the fly. If your platform contains a lot of user-generated video content, you cannot count on your users to also upload the SRT files for the videos they upload. Instead, your users depend on your platform to generate subtitles for their content. For instance, social-media users with numerous followers would expect that the app can and would generate subtitles for their uploaded videos and that, as soon as they turn on the subtitle feature, followers can immediately view the subtitles.

Cloudinary’s Google Video Transcript add-on enables you to automate the generation of speech-to-text transcripts of user-generated videos that have been uploaded to your site. For the best possible speech-recognition results, the add-on, which can translate videos in almost any language, applies powerful neural-network models to videos with Google's Cloud Speech API.

After a transcript becomes available, you can do the following:

  1. Automatically insert the transcript into your video in the form of subtitles.
  2. Parse the content of the transcript and display it on a webpage. That way, users can skim the content instead of watching the videos. This approach is extremely popular on learning platforms.

suptitle demo

Here’s the code for generating transcripts:

Copy to clipboard
cloudinary.v2.uploader.upload("learning_javascript.mp4", 
  { resource_type: "video", 
    raw_convert: "google_speech" },
  function(error, result) {console.log(result, error) });

Node.js

Once generation is complete, the add-on creates a new file with a .transcript extension in your Cloudinary account with the same public ID as your video file.

Note
If you include the notification_url parameter in your method call, the URL you specify receives a notification once the generation process concludes.

Depending on the size of the video file, the process might take a while.

You might wonder, “Why a .transcript file? Why not a .srt or .vtt file?” Well, you do have the option to have Cloudinary generate a .srt and a .vtt file. Just specify your preference, like this:

Copy to clipboard
cloudinary.v2.uploader.upload("learning_javascript.mp4", 
  { resource_type: "video", 
    raw_convert: "google_speech:srt:vtt" },
  function(error, result) {console.log(result, error) });

Node.js

When the process is complete, the add-on automatically uploads these three generated files to your account:

Copy to clipboard
.../raw/upload/learning_javascript.transcript
.../raw/upload/learning_javascript.srt
.../raw/upload/learning_javascript.vtt

Display of Generated Transcripts and Subtitles on Videos

The final step is to display the generated transcripts and subtitles on the video for delivery. It’s as simple as programmatically overlaying the files on the video, like this:

Copy to clipboard
cloudinary.video("learning_javascript", {overlay: {resource_type: "subtitles", public_id: "learning_javascript.transcript"}})

Node.js

The URL reads—

https://res.cloudinary.com/demo/video/upload/l_subtitles:learning_javascript.transcript/learning_javascript.mp4

References

Below are additional references:

Recent Blog Posts

Automatically Translating Videos for an International Audience

No matter your business focus—public service, B2B integration, recruitment—multimedia, in particular video, is remarkably effective in communicating with the audience. Before, making video accessible to diverse viewers involved tasks galore, such as eliciting the service of production studios to manually dub, transcribe, and add subtitles. Those operations were costly and slow, especially for globally destined content.

Read more
Cloudinary Helps Minted Manage Its Image-Generation Pipeline at Scale

David first shared his thoughts at our ImageCon coverence last October and this case study is an abbreviated version of Minted’s success using Cloudinary.

Over time, Faithful renderings of the creations of the illustrators, textile designers, painters, packaging designers, marketers, and stay-at-home moms, all of whom are core contributors of the Minted world, was getting harder and harder. Legacy technology wasn’t cutting it any more—and it was time for Cloudinary to step in.

Read more
Highlights on ImageCon 2021 and a Preview of ImageCon 2022

New year, same trend! Visual media will continue to play a monumental role in driving online conversions. To keep up with visual-experience trends and best practices, Cloudinary holds an annual conference called ImageCon, a one-of-a-kind event that helps attendees create the most engaging visual experiences possible.

Read more

New for DAM: Media Library Extension for Chrome

By Sharon Yelenik
A New Media Library Chrome Extension for Cloudinary DAM

With the introduction of the Media Library Extension, a Chrome-browser add-on that streamlines the access to, search for, and management of images and videos, Cloudinary offers yet another effective tool for its Digital Asset Management (DAM) solution. Let’s have a look at how most teams are currently working with media assets and how the new add-on not only boosts efficiency, but also renders the process a pleasure to work with.

Read more
New Features Supercharge Cloudinary’s Digital Asset Management Solution.

Today, I’m thrilled to announce the launch of Apps for Digital Asset Management and a Media Library Extension for the Chrome browser, which enables easy, flexible integration with all web-based applications in addition to making asset discovery more robust and accessible to all.

Read more
Scale and Automate Workflows With Modern Digital Asset Management Systems

With building, growing, and maintaining a strong digital presence being a top priority for all brands, high-quality visual content is paramount. In fact, consumers are 40 times more likely to share visual content on social networks than on other forums. Plus, a recent study from Wyzowl found that 84% of consumers made purchase decisions after watching a video, which explains why many brands are adding more and more visual media to their sites.

Read more