Cloudinary Blog

Muted Videos and Subtitles

Muted Videos and Subtitles

The bane of our existence is the lack of efficient ways for tackling the plethora of recurring tasks in our lives. One of those tasks is surfing the internet. We consume a lot of web content daily, of which a large percentage are images and videos. We’re constantly quickly scrolling through 30-second videos or checking out pictures of cute items we’d like to buy in our free time.

Some of those videos are in our native language; others are in foreign languages that we don’t understand. Sometimes we still enjoy the latter, however, because of the video’s graphical appeal. Other times, we might want to watch muted videos because of a loud immediate environment or an absence of headsets.

The Value of Subtitles

Subtitles are a superb feature for videos, muted or not.

Have you ever downloaded or watched someone download an SRT (.srt) file just to have the subtitles show up while watching a movie? As a software developer who builds apps that involve videos, you must inculcate the capacity that enables users to watch subtitles in videos.

Addition of Subtitles in Videos

How can you add video subtitles? Just as you can manually download and add SRT files to videos for subtitles, you can programmatically add subtitles to videos with Cloudinary. In fact, you can upload multiple SRT or WebVTT files to Cloudinary as normal raw files.

Making the content of those files show up as subtitles on your videos that have already been uploaded to Cloudinary is as straightforward as overlaying the subtitles on them. See this demo:

Ruby:
Copy to clipboard
cl_video_tag("dog", :overlay=>{:resource_type=>"subtitles", :public_id=>"sample_sub_en.srt"})
PHP v1:
Copy to clipboard
cl_video_tag("dog", array("overlay"=>array("resource_type"=>"subtitles", "public_id"=>"sample_sub_en.srt")))
PHP v2:
Copy to clipboard
(new VideoTag('dog.mp4'))
  ->overlay(Overlay::source(Source::subtitles('sample_sub_en.srt')));
Python:
Copy to clipboard
CloudinaryVideo("dog").video(overlay={'resource_type': "subtitles", 'public_id': "sample_sub_en.srt"})
Node.js:
Copy to clipboard
cloudinary.video("dog", {overlay: {resource_type: "subtitles", public_id: "sample_sub_en.srt"}})
Java:
Copy to clipboard
cloudinary.url().transformation(new Transformation().overlay(new SubtitlesLayer().publicId("sample_sub_en.srt"))).videoTag("dog");
JS:
Copy to clipboard
cloudinary.videoTag('dog', {overlay: new cloudinary.SubtitlesLayer().publicId("sample_sub_en.srt")}).toHtml();
jQuery:
Copy to clipboard
$.cloudinary.video("dog", {overlay: new cloudinary.SubtitlesLayer().publicId("sample_sub_en.srt")})
React:
Copy to clipboard
<Video publicId="dog" >
  <Transformation overlay={{publicId: "sample_sub_en.srt", resourceType: "subtitles"}} />
</Video>
Vue.js:
Copy to clipboard
<cld-video publicId="dog" >
  <cld-transformation :overlay="{publicId: 'sample_sub_en.srt', resourceType: 'subtitles'}" />
</cld-video>
Angular:
Copy to clipboard
<cl-video public-id="dog" >
  <cl-transformation overlay="subtitles:sample_sub_en.srt">
  </cl-transformation>
</cl-video>
.NET:
Copy to clipboard
cloudinary.Api.UrlVideoUp.Transform(new Transformation().Overlay(new SubtitlesLayer().PublicId("sample_sub_en.srt"))).BuildVideoTag("dog")
Android:
Copy to clipboard
MediaManager.get().url().transformation(new Transformation().overlay(new SubtitlesLayer().publicId("sample_sub_en.srt"))).resourceType("video").generate("dog.mp4");
iOS:
Copy to clipboard
cloudinary.createUrl().setResourceType("video").setTransformation(CLDTransformation().setOverlay("subtitles:sample_sub_en.srt")).generate("dog.mp4")

Simply specify the overlay parameter l_subtitles, followed by the public ID of the raw file and its extension. That syntax works in the URL transformation as shown above.

Programmatically, you can code it this way:

Copy to clipboard
cloudinary.video("dog", {overlay: {resource_type: "subtitles", public_id: "sample_sub_en.srt"}})

Node.js

Furthermore, you can customize the way subtitles appear in the video: the color, appearance, font, etc. For example, here’s how to make subtitles yellow in color with a red background and a larger font:

Copy to clipboard
cloudinary.video("dog", {overlay: {font_family: "arial", font_size: 100, resource_type: "subtitles", public_id: "sample_sub_en.srt"}, color: "#ffff00", background: "red"})

Node.js

Generation of Transcripts and Subtitles for Videos

Another important step is to generate transcripts for your videos on the fly. If your platform contains a lot of user-generated video content, you cannot count on your users to also upload the SRT files for the videos they upload. Instead, your users depend on your platform to generate subtitles for their content. For instance, social-media users with numerous followers would expect that the app can and would generate subtitles for their uploaded videos and that, as soon as they turn on the subtitle feature, followers can immediately view the subtitles.

Cloudinary’s Google Video Transcript add-on enables you to automate the generation of speech-to-text transcripts of user-generated videos that have been uploaded to your site. For the best possible speech-recognition results, the add-on, which can translate videos in almost any language, applies powerful neural-network models to videos with Google's Cloud Speech API.

After a transcript becomes available, you can do the following:

  1. Automatically insert the transcript into your video in the form of subtitles.
  2. Parse the content of the transcript and display it on a webpage. That way, users can skim the content instead of watching the videos. This approach is extremely popular on learning platforms.

suptitle demo

Here’s the code for generating transcripts:

Copy to clipboard
cloudinary.v2.uploader.upload("learning_javascript.mp4", 
  { resource_type: "video", 
    raw_convert: "google_speech" },
  function(error, result) {console.log(result, error) });

Node.js

Once generation is complete, the add-on creates a new file with a .transcript extension in your Cloudinary account with the same public ID as your video file.

Note
If you include the notification_url parameter in your method call, the URL you specify receives a notification once the generation process concludes.

Depending on the size of the video file, the process might take a while.

You might wonder, “Why a .transcript file? Why not a .srt or .vtt file?” Well, you do have the option to have Cloudinary generate a .srt and a .vtt file. Just specify your preference, like this:

Copy to clipboard
cloudinary.v2.uploader.upload("learning_javascript.mp4", 
  { resource_type: "video", 
    raw_convert: "google_speech:srt:vtt" },
  function(error, result) {console.log(result, error) });

Node.js

When the process is complete, the add-on automatically uploads these three generated files to your account:

Copy to clipboard
.../raw/upload/learning_javascript.transcript
.../raw/upload/learning_javascript.srt
.../raw/upload/learning_javascript.vtt

Display of Generated Transcripts and Subtitles on Videos

The final step is to display the generated transcripts and subtitles on the video for delivery. It’s as simple as programmatically overlaying the files on the video, like this:

Copy to clipboard
cloudinary.video("learning_javascript", {overlay: {resource_type: "subtitles", public_id: "learning_javascript.transcript"}})

Node.js

The URL reads—

https://res.cloudinary.com/demo/video/upload/l_subtitles:learning_javascript.transcript/learning_javascript.mp4

References

Below are additional references:

Recent Blog Posts

Create Lightweight Sites With Low-Code and No-Code Technology

Consumers expect modern websites to be mainly visual. But, the more compelling and complex the related media is, the more data is involved, compounding the site’s weight. In today’s content-craving world, delivering unoptimized media can cost you because it leads to sluggish page loads, resulting in visitors abandoning your site in search of a faster alternative. In fact, a page load that takes more than three seconds can cause as many as 40% of your visitors to bounce. Given this competitive, digital-first environment, you can’t afford to lose page views, for time is of the essence.

Read more
A Blueprint for AWS-Secured Webhook Listeners for Cloudinary

tl;dr: An AWS-secured and optimized Cloudinary webhook listener for extending the Cloudinary service

Code: Github

A webhook is a communication medium for sending notifications from one platform to another about events that occurred. In place are user-defined HTTP callbacks that are triggered by specific events. When a triggered event takes place on the source site, the webhook listens to the event, collects the data, and sends it to the URL you specified in the form of an HTTP request.

Read more
New Accessibility Features for Cloudinary’s Product Gallery Widget

Cloudinary’s Product Gallery widget, which launched in 2019, has enabled many brands to effectively and efficiently showcase their products in a sleek and captivating manner, saving countless hours of development time and accelerating release cycles. By adding Cloudinary’s Product Gallery widget with its customizable UI to their product page, retailers reap numerous benefits, often turning visitors into customers in short order.

Read more
Why Successful Businesses Engage With and Convert Audiences With Visual Media

Most business buyers prefer to research purchase options online, as do many shoppers. No wonder online retail sales in the U.S. rose by 32.4% in 2020—an impressive gain of $105 billion.

For B2B and B2C businesses, text-heavy websites are no longer adequate in attracting shoppers. Instead, engaging visual media—spin images, videos, 3D models, augmented reality—are becoming a must for conveying eye-catching details and differentiators about products or services.

Read more
Making User-Generated Content (UGC) Shoppable With Cloudinary

User-generated content (UGC) is a powerful marketing tool. Not only does video complement marketing efforts for e-commerce by enabling customers to explore products in greater detail, but UGC also adds an element of trust. As a bonus, user-generated video is an exceptional opportunity for e-businesses to attract website traffic without their marketing team having to create promotional videos from scratch. User-generated content drives conversions and brand loyalty as a direct result of authentic interaction.

Read more