Cloudinary Blog

New Google-powered add-on for automatic video categorization and tagging

Introducing Automatic Video Tagging and Content Categorization

Due to significant growth of the web and improvements in network bandwidth, video is now a major source of information and entertainment shared over the internet. As a developer or asset manager, making corporate videos available for viewing, not to mention user-uploaded videos, means you also need a way to categorize them according to their content and make your video library searchable. Most systems end up organizing their video by metadata like the filename, or with user-generated tags (e.g., youtube). This sort of indexing method is subjective, inconsistent, time-consuming, incomplete and superficial.

A well-organized indexing system lets you easily manage and organize your media libraries:

  • Enable personnel across your entire organization to find resources they may need
  • Increase engagement by helping your users find exactly what they’re looking for
  • Help you connect your users with common interests and help them find other content that would interest them
  • Increase sales or advertising revenue by determining the main subjects that interest particular users and integrating this information with your existing analytics/personalization tools to display relevant product recommendations or adverts

But ultimately, any sort of manual video categorization process would take huge amounts of time and resources.

Webinar
Delivering Compelling Video Experiences at Scale

Introducing Cloudinary's Automatic Video Tagging add-on, powered by Google Cloud Video Intelligence, which is now fully integrated into Cloudinary's video management and delivery pipeline. State-of-the-art machine learning allows for the recognition of various visual objects and concepts in videos, simplifying and automating the categorization and tagging process.

Using the Automatic Video Tagging add-on

Take a look at the following video of horses:

Using the add-on, automatically assigning resource tags to the video is as simple as adding 2 parameters when either uploading a new video or updating an existing video: set the categorization parameter to google_video_tagging and the auto_tagging parameter to the minimum confidence score necessary before automatically adding a detected category as a tag. For example, uploading the horses video and requesting automatic tagging for all categories meeting a confidence score of over 40%:

Ruby:
Copy to clipboard
Cloudinary::Uploader.upload("horses.mp4", 
   :resource_type => :video, :categorization => "google_video_tagging", :auto_tagging => 0.4)
PHP:
Copy to clipboard
\Cloudinary\Uploader::upload("horses.mp4", 
  array("categorization" => "google_video_tagging", "auto_tagging" => 0.4));
Python:
Copy to clipboard
cloudinary.uploader.upload("horses.mp4",
  categorization = "google_video_tagging", auto_tagging = 0.4)
Node.js:
Copy to clipboard
cloudinary.uploader.upload("horses.mp4", 
  function(result) { console.log(result); }, 
  { categorization: "google_video_tagging", auto_tagging: 0.4 });
Java:
Copy to clipboard
cloudinary.uploader().upload("horses.mp4", ObjectUtils.asMap(
  "categorization", "google_video_tagging", "auto_tagging", "0.4"));

Once the categorization process completes, the information is returned to Cloudinary and all categories that exceed your specified confidence score are automatically added as tags on your video.

stallion.jpg

Below is a snippet of the upload response for the horse video:

Copy to clipboard
{
...
tags"=>  ["animal", "freezing", "frost", "horse",  … ],
"info": {
   "google_video_tagging": {
      "status": "complete",
      "data": [
         [{"tag": "horse", 
          "start_time_offset": 0.0, 
          "end_time_offset": 12.6364, 
          "confidence": 0.8906},
          {"tag": "horse", 
          "start_time_offset": -1, 
          "end_time_offset": -1, 
          "confidence": 0.8906},
          {"tag": "animal", 
          "start_time_offset": 0.0, 
          "end_time_offset": 13.47364, 
          "confidence": 0.8906},

          ]
...

The benefits of video tagging

As can be seen in the example snippet above, various categories were automatically detected in the uploaded video and automatically added as tags. Each category is listed together with the start and end times of the relevant video segment (an offset time of -1 means the category represents the entire video) and the confidence score of the detected category, where 1.0 means 100% confidence.

Once the video has been categorized, that information can be shared with your analytics tools. Cross-examining both the categorization and usage data can yield valuable insights into how different videos impact engagement and conversion. Do the videos show indoor or outdoor scenes? Do they include people? Animals? This information can then be leveraged for AB testing and user profiling.

For example, you can test how different videos, (e.g., with or without animals) may impact engagement for a specific product or service, helping you utilize the optimal content when designing websites, apps or email campaigns. You may determine that a user watching videos of parties, events, sports, and music is probably a college student or young adult, whereas a user that uploads videos of parks, children, and playgrounds is more likely to be a parent. This knowledge can help you focus your content on the right audience and increase engagement and conversion.

Additionally, a well indexed, organized library of videos can be leveraged across your entire organization. Tagging is particularly useful if your company has a constantly growing library of digital assets that need to be made available for various teams within your organization. For example, if the marketing team needs a video of a dog for an email campaign, they can search for and select the most appropriate video.

See automatic tagging in action Visit Cloudinary's Video Transcoding demo where you can check out the results of the automatic tagging add-on for a number of sample videos or even upload your own. You can also see examples of a variety of advanced video transformations as well as a demonstration of the Video Transcription add-on.

Summary

The Google-powered Automatic Video Tagging add-on provides you with meaningful data extracted from videos. Take advantage of that data to make strategic business decisions that could improve your users’ experience and drive greater profits. Cloudinary’s service, together with the fully integrated Automatic Video Tagging add-on, provides you with the powerful ability to streamline your content management as well as increase your users’ engagement and conversion.

video tagging

The add-on is available with all Cloudinary plans and offers a free add-on tier for you to try out. If you don't have a Cloudinary account yet, sign up for a free account.

About Cloudinary

Cloudinary provides easy-to-use, cloud-based media management solutions for the world’s top brands. With offices in the US, UK and Israel, Cloudinary has quickly become the de facto solution used by developers and marketers at major companies around the world to streamline rich media management and deliver optimal end-user experiences.

For more information, visit www.cloudinary.com or follow us on Twitter.


Further Reading on Video Manipulation

Recent Blog Posts

Automation Frees Up PetRescue’s Staff to Help Pets Find Their Forever Homes

As we spend more time at home, many of us are adopting pets for the joy, companionship and a surprising range of health benefits. In Australia, where our nonprofit customer PetRescue is located, there’s a shortage of pets to adopt. Last August, the Guardian reported that dog shelters in Australia emptied and adoption fees for puppies were running as high as $AUS1800.

Read more
Cloudinary and Contentful Make Modern Content Management Easier

I am pleased to share that Cloudinary and Contentful have joined forces to further streamline the creation, processing, and delivery of online content through Cloudinary’s digital asset management (DAM) solution and advanced transformation and delivery capabilities for images and video. What’s more, the partnership delivers a headless approach to DAM. By leveraging APIs for media management tasks, marketers and developers alike benefit from an integrated stack of optimized assets for optimization and automation. As a result, page loads are fast and beautiful, and at scale—with less overhead and effort.

Read more
Introducing Cloudinary's Nuxt Module

Since its initial release in October 2016 by the Chopin brothers as a server-side framework that runs on top of Vue.js, Nuxt (aka Nuxt.js) has gained prominence in both intuitiveness and performance. The framework offers numerous built-in features based on a modular architecture, bringing ease and simplicity to web development. Not surprisingly, Nuxt.js has seen remarkable growth in adoption by the developer community along with accolades galore. At this writing, Nuxt has earned over 30K stars on GitHub and 96 active modules with over a million downloads per month. And the upward trend is ongoing.

Read more
How Quality and Quantity can go Hand in Hand

When it comes to quality versus quantity, you’ll often hear people say, “It’s the quality that counts, not the quantity”. While that’s true in many situations, there are also cases where you want both quality and quantity. You may have thousands of images on your website and you want them all to look great. This is especially important if your website allows users to upload their own content, for example, to sell their own products or services. You don't want their poor quality images to reflect badly on your brand.

Read more
Product Videos 101: What Makes Them Great?

A product’s benefits and usage, including its value proposition, features, and instructive details, are best demonstrated through video. Product-video types vary, depending on the funnel, channel, and audience, the most popular ones being demos, reviews, installation, and how-tos.

Read more