Cloudinary Blog

Automatically Crop Videos Without Losing Focus

Content-Aware Automatic Cropping for Video

Delivering videos according to the aspect ratios defined by social media for multiple devices and platforms is a growing challenge. The continuously rising volume of vertical videos and the corresponding increase in video traffic on mobile devices (now up to 57% of online videos watched) have only exacerbated the situation, with no letup in sight.

Reality is, smartphone users hold their phone vertically while scrolling or browsing online content, including videos that are not optimized for mobile devices. It simply doesn’t occur to those users to switch to horizontal mode for an enhanced view, or it doesn't feel worth their effort. This means that providers of online content must make sure their videos accommodate all viewing devices, ideally through automation.


Sign up for Cloudinary free today!


Cloudinary, which for years has been performing automated cropping of images for size economy and compliance with aspect ratios, can now do the same for videos, adding to its already powerful video solution. Impressively, Cloudinary’s content-aware cropping capability identifies the must-keep sections by analyzing videos through advanced AI and machine-learning techniques, churning out intelligently downsized videos in any aspect ratio. It’s as simple as setting the crop mode to fill and the gravity to auto.

Ruby:
Copy to clipboard
cl_video_tag("ship", :width=>300, :gravity=>"auto", :aspect_ratio=>"1:1", :crop=>"fill")
PHP v1:
Copy to clipboard
cl_video_tag("ship", array("width"=>300, "gravity"=>"auto", "aspect_ratio"=>"1:1", "crop"=>"fill"))
PHP v2:
Copy to clipboard
(new VideoTag('ship.mp4'))
  ->resize(Resize::fill()->width(300)
    ->aspectRatio(AspectRatio::ar1X1())
    ->gravity(Gravity::autoGravity()));
Python:
Copy to clipboard
CloudinaryVideo("ship").video(width=300, gravity="auto", aspect_ratio="1:1", crop="fill")
Node.js:
Copy to clipboard
cloudinary.video("ship", {width: 300, gravity: "auto", aspect_ratio: "1:1", crop: "fill"})
Java:
Copy to clipboard
cloudinary.url().transformation(new Transformation().width(300).gravity("auto").aspectRatio("1:1").crop("fill")).videoTag("ship");
JS:
Copy to clipboard
cloudinary.videoTag('ship', {width: 300, gravity: "auto", aspectRatio: "1:1", crop: "fill"}).toHtml();
jQuery:
Copy to clipboard
$.cloudinary.video("ship", {width: 300, gravity: "auto", aspect_ratio: "1:1", crop: "fill"})
React:
Copy to clipboard
<Video publicId="ship" >
  <Transformation width="300" gravity="auto" aspectRatio="1:1" crop="fill" />
</Video>
Vue.js:
Copy to clipboard
<cld-video publicId="ship" >
  <cld-transformation width="300" gravity="auto" aspectRatio="1:1" crop="fill" />
</cld-video>
Angular:
Copy to clipboard
<cl-video public-id="ship" >
  <cl-transformation width="300" gravity="auto" aspect-ratio="1:1" crop="fill">
  </cl-transformation>
</cl-video>
.NET:
Copy to clipboard
cloudinary.Api.UrlVideoUp.Transform(new Transformation().Width(300).Gravity("auto").AspectRatio("1:1").Crop("fill")).BuildVideoTag("ship")
Android:
Copy to clipboard
MediaManager.get().url().transformation(new Transformation().width(300).gravity("auto").aspectRatio("1:1").crop("fill")).resourceType("video").generate("ship.mp4");
iOS:
Copy to clipboard
cloudinary.createUrl().setResourceType("video").setTransformation(CLDTransformation().setWidth(300).setGravity("auto").setAspectRatio("1:1").setCrop("fill")).generate("ship.mp4")

Rotate or Move On: The Challenge with Video on Mobile

Nearly half of all smartphone users watch short-form videos or live posts at least once a week. Because most people hold their phones vertically for comfort, yet videos are best viewed horizontally, many folks shy away from the trouble of rotating their phone and waiting for the video to update. The solution? Have videos play with the correct aspect ratio, eliminating the rotation step and avoiding accidental dropping of the phone.

Additionally, given the monumental role of social media in video consumption, you must deliver videos on those platforms across all devices. The chart below showing the various aspect ratios required for optimally delivering videos for Facebook and Instagram, clearly demonstrates the challenge, especially if large quantities of videos are involved.

Introducing Automatic Content-Aware Cropping

You can now automate all your video cropping with Cloudinary’s automatic content-aware automatic cropping functionality. No more time consuming manual editing. Cloudinary has developed a complex deep-learning algorithm that analyzes frames from the video and identifies where the most interesting areas are. From this analysis, a heatmap is produced and then used to crop the video intelligently.

The cropped video will now follow the most interesting area of the video throughout the duration, ensuring that your videos retain all the important features you wanted when you filmed them, while filling the screen no matter the aspect ratio.

As we mentioned above, all you need to do is set the crop mode to fill and the gravity to auto. This will begin the process of analysing the video. We recommend performing this on upload as an eager transformation to ensure the analysis is complete before delivery (or alternatively using the explict method before delivery on a video already in your account). Once that analysis has been performed on a specific video in your account, you can deliver your content-aware video in any aspect ratio you need; Landscape, Square, Facebook Vertical, Instagram Story Vertical.

Automatic Cropping Versus Default Cropping

Let’s take a look at how content-aware automatic cropping compares with the standard center crop. Notice how the default crop only has the dog in shot for a short period of time when he passes through the center of the frame. With automatic cropping, this impressive frisbie catch stays in shot throughout the video, despite starting on one side of the frame and finishing on the other.

Click any video to see the comparison in action.

Original video
Default crop
(Center gravity)
Automatic crop

Taking it Further

We could use this new functionality to extend the existing mobile video experience and actually serve up a landscape or vertical video depending on the orientation of the device. If the user lands on your page whilst browsing in portrait orientation we can set our HTML5 video source to our vertically cropped video. If they then decide to switch to landscape then we change the source to the original landscape video. Full screen video for both orientations.

See the Pen Orientation switch demo by Cloudinary (@Cloudinary) on CodePen.


It's also super easy to combine content-aware cropping with a number of other great features of Cloudinary’s Dynamic Video Platform. Take a look at how to use automatic video transcoding and content-aware compression to achieve complete end-to-end automation and optimization.

In Summary

So there you have it, Cloudinary’s powerful video solution just got even more powerful. Advanced AI and deep learning algorithms enable you to deliver your videos in the right size and aspect ratio, for every platform or device and automate the whole process. Go ahead and try it out with one of your own videos on our demo page, or you can sign up for a free account and get going with the full product. Just remember to set the crop mode to fill and the gravity to auto. Check out our documentation for more info.


Further Reading on Video Manipulation

Recent Blog Posts

Automatically Translating Videos for an International Audience

No matter your business focus—public service, B2B integration, recruitment—multimedia, in particular video, is remarkably effective in communicating with the audience. Before, making video accessible to diverse viewers involved tasks galore, such as eliciting the service of production studios to manually dub, transcribe, and add subtitles. Those operations were costly and slow, especially for globally destined content.

Read more
Cloudinary Helps Minted Manage Its Image-Generation Pipeline at Scale

David first shared his thoughts at our ImageCon coverence last October and this case study is an abbreviated version of Minted’s success using Cloudinary.

Over time, Faithful renderings of the creations of the illustrators, textile designers, painters, packaging designers, marketers, and stay-at-home moms, all of whom are core contributors of the Minted world, was getting harder and harder. Legacy technology wasn’t cutting it any more—and it was time for Cloudinary to step in.

Read more
Highlights on ImageCon 2021 and a Preview of ImageCon 2022

New year, same trend! Visual media will continue to play a monumental role in driving online conversions. To keep up with visual-experience trends and best practices, Cloudinary holds an annual conference called ImageCon, a one-of-a-kind event that helps attendees create the most engaging visual experiences possible.

Read more

New for DAM: Media Library Extension for Chrome

By Sharon Yelenik
A New Media Library Chrome Extension for Cloudinary DAM

With the introduction of the Media Library Extension, a Chrome-browser add-on that streamlines the access to, search for, and management of images and videos, Cloudinary offers yet another effective tool for its Digital Asset Management (DAM) solution. Let’s have a look at how most teams are currently working with media assets and how the new add-on not only boosts efficiency, but also renders the process a pleasure to work with.

Read more
New Features Supercharge Cloudinary’s Digital Asset Management Solution.

Today, I’m thrilled to announce the launch of Apps for Digital Asset Management and a Media Library Extension for the Chrome browser, which enables easy, flexible integration with all web-based applications in addition to making asset discovery more robust and accessible to all.

Read more
Scale and Automate Workflows With Modern Digital Asset Management Systems

With building, growing, and maintaining a strong digital presence being a top priority for all brands, high-quality visual content is paramount. In fact, consumers are 40 times more likely to share visual content on social networks than on other forums. Plus, a recent study from Wyzowl found that 84% of consumers made purchase decisions after watching a video, which explains why many brands are adding more and more visual media to their sites.

Read more