Cloudinary Blog

Video tagging made easy: Google-powered add-on for automatic video tagging

Video tagging made easy: Google-powered add-on for automatic video tagging

Due to significant growth of the web and improvements in network bandwidth, video is now a major source of information and entertainment shared over the internet. If you are managing a library of corporate videos, user-uploaded videos, or even your own personal videos, you need a way to categorize them according to their content and make your video library searchable.

Many organizations organize their video by metadata like the filename, or with user-generated tags (e.g., youtube video tags). This sort of manual indexing method is ineffective, because it is subjective to the user adding the tags, incomplete, and time consuming. What you need is a way to automatically tag videos, accurately and consistently.

An automated video tagging system lets you easily manage and organize your videos:

  • Enable in-house teams to find video resources they need
  • Increase engagement by helping users easily find videos they’re looking for
  • Help you connect your users with common interests and help them find more content that would interest them
  • Increase sales or advertising revenue by determining the main subjects that interest of your users, and using it to display relevant product recommendations or ads

Introducing Cloudinary's Automatic Video Tagging add-on, powered by Google Cloud Video Intelligence, which is now fully integrated into Cloudinary's video management and delivery platform.

Google’s state-of-the-art machine learning can recognize visual objects and concepts in videos with surprisingly high accuracy. You can leverage the power of Google to tag a large number of videos, instantly and automatically.

Using the Automatic Video Tagging add-on

Take a look at the following video of horses:

Using the add-on, automatically assigning resource tags to the video is as simple as adding 2 parameters when either uploading a new video or updating an existing video:

  • Set the categorization parameter to google_video_tagging
  • Set the auto_tagging parameter to the minimum confidence score you are willing to accept. Any tag above this score will be automatically added as a tag.

For example, uploading the horses video and requesting automatic tagging for all categories meeting a confidence score of over 40%:

Ruby:
Copy to clipboard
Cloudinary::Uploader.upload("horses.mp4", 
   :resource_type => :video, :categorization => "google_video_tagging", :auto_tagging => 0.4)
PHP:
Copy to clipboard
\Cloudinary\Uploader::upload("horses.mp4", 
  array("categorization" => "google_video_tagging", "auto_tagging" => 0.4));
Python:
Copy to clipboard
cloudinary.uploader.upload("horses.mp4",
  categorization = "google_video_tagging", auto_tagging = 0.4)
Node.js:
Copy to clipboard
cloudinary.uploader.upload("horses.mp4", 
  function(result) { console.log(result); }, 
  { categorization: "google_video_tagging", auto_tagging: 0.4 });
Java:
Copy to clipboard
cloudinary.uploader().upload("horses.mp4", ObjectUtils.asMap(
  "categorization", "google_video_tagging", "auto_tagging", "0.4"));

Once the categorization process completes, the information is returned to Cloudinary and all categories that exceed your specified confidence score are automatically added as tags on your video.

stallion.jpg

Below is a snippet of the upload response for the horse video:

Copy to clipboard
{
...
tags"=>  ["animal", "freezing", "frost", "horse",  … ],
"info": {
   "google_video_tagging": {
      "status": "complete",
      "data": [
         [{"tag": "horse", 
          "start_time_offset": 0.0, 
          "end_time_offset": 12.6364, 
          "confidence": 0.8906},
          {"tag": "horse", 
          "start_time_offset": -1, 
          "end_time_offset": -1, 
          "confidence": 0.8906},
          {"tag": "animal", 
          "start_time_offset": 0.0, 
          "end_time_offset": 13.47364, 
          "confidence": 0.8906},

          ]
...

See automatic tagging in action!

Visit Cloudinary's Video Transcoding demo where you can check out the results of the automatic tagging add-on for a number of sample videos or even upload your own. You can also see examples of a variety of advanced video transformations as well as a demonstration of the Video Transcription add-on.

The benefits of video tagging

As can be seen in the example snippet above, various categories were automatically detected in the uploaded video and automatically added as tags. Each category is listed together with the start and end times of the relevant video segment (an offset time of -1 means the category represents the entire video) and the confidence score of the detected category, where 1.0 means 100% confidence.

Once the video has been categorized, that information can be shared with your analytics tools. Cross-examining both the categorization and usage data can yield valuable insights into how different videos impact engagement and conversion. Do the videos show indoor or outdoor scenes? Do they include people? Animals? This information can then be leveraged for AB testing and user profiling.

For example, you can test how different videos, (e.g., with or without animals) may impact engagement for a specific product or service, helping you utilize the optimal content when designing websites, apps or email campaigns. You may determine that a user watching videos of parties, events, sports, and music is probably a college student or young adult, whereas a user that uploads videos of parks, children, and playgrounds is more likely to be a parent. This knowledge can help you focus your content on the right audience and increase engagement and conversion.

Additionally, a well indexed, organized library of videos can be leveraged across your entire organization. Tagging is particularly useful if your company has a constantly growing library of digital assets that need to be made available for various teams within your organization. For example, if the marketing team needs a video of a dog for an email campaign, they can search for and select the most appropriate video.

Leveraging video tagging for user-generated content (UGC)

User-generated content can greatly contribute to your website, you cannot count on users to tag and describe their videos. For instance, if you’re running a real-estate website, users would upload videos of their property but might not accurately tag all the elements in the video.

Instead, you can rely on Cloudinary’s auto-tagging to find objects like "pool" and "palm tree," which certain house shoppers might be looking for. Those tags greatly improve the listing’s discoverability and searchability.

Another example: Consider an animal shelter that posts videos of animals that are up for adoption. Instead of having to manually type in a comprehensive set of tags for each animal, the staff needs to enter only the basic information, based on which the auto-tagging capability can figure out the animal type (like cat, dog, hamster, or horse), color, and other attributes.

Any web app into which users can upload their media can benefit from the Cloudinary API, saving you the time and effort of building the capabilities yourself: upload, transformation, storage, and so on.

Summary

The Google-powered Automatic Video Tagging add-on provides you with meaningful data extracted from videos. Take advantage of that data to make strategic business decisions that could improve your users’ experience and drive greater profits. Cloudinary’s service, together with the fully integrated Automatic Video Tagging add-on, provides you with the powerful ability to streamline your content management as well as increase your users’ engagement and conversion.

video tagging

The add-on is available with all Cloudinary plans and offers a free add-on tier for you to try out. If you don't have a Cloudinary account yet, sign up for a free account.

Recent Blog Posts

Automatically Translating Videos for an International Audience

No matter your business focus—public service, B2B integration, recruitment—multimedia, in particular video, is remarkably effective in communicating with the audience. Before, making video accessible to diverse viewers involved tasks galore, such as eliciting the service of production studios to manually dub, transcribe, and add subtitles. Those operations were costly and slow, especially for globally destined content.

Read more
Cloudinary Helps Minted Manage Its Image-Generation Pipeline at Scale

David first shared his thoughts at our ImageCon coverence last October and this case study is an abbreviated version of Minted’s success using Cloudinary.

Over time, Faithful renderings of the creations of the illustrators, textile designers, painters, packaging designers, marketers, and stay-at-home moms, all of whom are core contributors of the Minted world, was getting harder and harder. Legacy technology wasn’t cutting it any more—and it was time for Cloudinary to step in.

Read more
Highlights on ImageCon 2021 and a Preview of ImageCon 2022

New year, same trend! Visual media will continue to play a monumental role in driving online conversions. To keep up with visual-experience trends and best practices, Cloudinary holds an annual conference called ImageCon, a one-of-a-kind event that helps attendees create the most engaging visual experiences possible.

Read more

New for DAM: Media Library Extension for Chrome

By Sharon Yelenik
A New Media Library Chrome Extension for Cloudinary DAM

With the introduction of the Media Library Extension, a Chrome-browser add-on that streamlines the access to, search for, and management of images and videos, Cloudinary offers yet another effective tool for its Digital Asset Management (DAM) solution. Let’s have a look at how most teams are currently working with media assets and how the new add-on not only boosts efficiency, but also renders the process a pleasure to work with.

Read more
New Features Supercharge Cloudinary’s Digital Asset Management Solution.

Today, I’m thrilled to announce the launch of Apps for Digital Asset Management and a Media Library Extension for the Chrome browser, which enables easy, flexible integration with all web-based applications in addition to making asset discovery more robust and accessible to all.

Read more
Scale and Automate Workflows With Modern Digital Asset Management Systems

With building, growing, and maintaining a strong digital presence being a top priority for all brands, high-quality visual content is paramount. In fact, consumers are 40 times more likely to share visual content on social networks than on other forums. Plus, a recent study from Wyzowl found that 84% of consumers made purchase decisions after watching a video, which explains why many brands are adding more and more visual media to their sites.

Read more