Cloudinary Blog

Handling Media at Scale Using Automation and AI

Managing Media at Scale With Automation and AI

In today’s digital world, engaging online content is key for catching audience attention. At ImageCon 2020, Cloudinary’s annual conference, Yaron Reichert and me, Jackie Rosenzveig, hosted a session titled ’Handling Media at Scale Using Automation and AI’, which delved into the following:

  • The importance of creating and maintaining compelling visual content through automation with a focus on four major developer tasks
  • The roles AI and automation play in every stage of the content lifecycle, including enhancements and optimization at scale, for a strong user experience

Below is the video of the full session.

Read on for a quick recap of all the cool AI and automation features our session covers:

Media Asset Lifecycle

In the first part of our presentation, we review the standard lifecycle of an image or a video:

  • Creation: The birth of an image from a photo shoot or graphic design.
  • Editing: Fine-tuning for the purpose of matching a website layout, complying with brand guidelines, and catering to audience preferences.
  • Delivery: Optimization and display that align with the user’s device, network condition, and other factors.
  • Enhancement: Analyses on the degree of success of the content along with insight on areas that need improvement.

AI and automation can play a masterful part in all four of these stages, enabling you to deliver rich and scalable media in a seamless and efficient end-to-end process—with no need for specialized manpower or graphic editing software.

For the remainder of our session, we walk you through four very popular use-cases to show how AI and automation can help speed up and simplify your end-to-end media needs:

Automating Social-Video Creation

With brands relying more and more on social media to reach their target audience, the demand for social videos has skyrocketed. Research consistently shows that videos foster stronger customer engagement and dialog than other content types.

Video Engages and Converts Customers

Even though you can’t predict which social video will be the next to go viral, it's important to keep in mind that all well-crafted and successful videos have a catchy title and are brand consistent. They also invariably apply the appropriate aspect ratio for the social network and contain easy-to-read captions.

In this session, Yaron demonstrates how Cloudinary's sophisticated AI-driven "g_auto" algorithms can automatically crop your videos to fit the various platforms, in particular cropping landscape videos to the required square and portrait aspect ratios, while keeping the main subject in the frame throughout the video!

He also talks about the importance of including titles and captions to capture the attention of the large percentage of your audience who may start watching your videos on mute, the value of a modern progress bar or frame to give users an idea of video progress, and consistent branding in the form of recognizable logos, fonts, and colors. Cloudinary makes it easy to add these programmatically to all of your videos with simple transformation parameters.

Enhancing the reliability and user experience of User-Generated Content (UGC)

Social-media users love viewing one another’s comments, opinions, and the like, but only if those contributions are visually appealing. On the average, industries enjoy a rise of 161 percent in the conversion rate of page visitors who see UGC and those who don’t. However, tremendous room for error exists in UGC, such as design mismatches and noncompliance with the business’s quality or security standards.

In the second part of our session, I got to show how you can "maximize the experience while minimizing the surprises of UGC with, you guessed it -- AI and automation -- including:

  • Evaluate images for quality and then flag (or reject) over-pixelated images
  • Check whether an uploaded image will work well for your color-blind users .
  • Scan for malware and inappropriate content
  • Automatically crop uploaded images around a specific object type or product category
  • Automatically generate image tags for SEO
  • Capture metadata from users
  • Enhance visuals with stylistic filters, cool effects, and other visual adjustments

UGC Slide

Optimizing Videos

Today’s online videos must load quickly, play smoothly, and look impressive; otherwise, visitors will move on.

High-quality videos exact a trade-off between quality, size, speed, and the cost of the compression process. Additionally, different devices and browsers require different formats and codecs. Applying adjustments around bitrate, keyframe intervals and frame rate can all impact the visual quality and bandwidth required to deliver your videos.

Complex video

In part 3 of our session, Yaron points out that attempting to create one video that fits all formats and devices is not the answer because this would require going for the lowest common denominator. On the other hand, as browser formats change and new capabilities emerge, manually formatting videos to accommodate all the platforms and devices would be a nearly impossible task.

AI coupled with automation is the right path to take for the capability to dynamically pinpoint the “sweet spot” for optimal bandwidth, speed, and quality for any video on any platform. Yaron shows how to take advantage of Cloudinary's automatic format and quality selection features (aka f_auto and q_auto) to ensure you are always delivering that perfect balance of visual excellence and great performance to every viewer.

Programmatically Managing Media via CLI

Many businesses maintain a large repository of media assets. Managing them includes operations like criteria-based searches, bulk operations on assets, as well as creating and modifying various media entities that you need for your apps. Even though Cloudinary has offered these capabilities pretty much from the start (via its Admin and Upload APIs), these operations usually required you to have a backend application.

In the last few minutes of our session, I walk you through the 3 quick steps involved in setting up and running commands with Cloudinary’s new command-line interface (CLI). Using it, you can run the same administration and upload operations that our server-side SDKs and REST APIs offer, with simple CLI commands. But wait, there's more! The CLI also comes with several special helper methods to help you do things like migrate external files to Cloudinary, sync files between a local folder and Cloudinary or quickly generate the code for any of our embeddable widgets. Super cool stuff!

CLI slide

Bottom line, the CLI means that programmatically organizing and optimizing your extensive media catalogs can be smooth, reliable, fast, and even easy!

The Power of AI and Automation for Media

Whether you are optimizing your own social content, UGC, or video; or managing media internally within your organization, you are probably already painfully aware of the common pitfalls many businesses face today in working with manual processes, especially as your media needs and your application reach scales.

The many features this ImageCon session demonstrates as it walks through the 4 super-common use-cases we described above, have one thing common. They all involve complex AI and automation algorithms that you can easily activate with single lines of code e, to handle your media needs, regardless of scale.

Cloudinary’s user-oriented offerings were developed through countless hours of research, deep learning, and coding. We’re dedicated to helping you exceed customer expectations, which will continue to rise.

Recent Blog Posts

Amplify Your Jamstack With Video

By Alex Patterson
Amplify Your Jamstack With Cloudinary Video

As defined by Amazon Web Services (AWS), Amplify is a set of products and tools with which mobile and front-end web developers can build and deploy AWS-powered, secure, and scalable full-stack apps. Also, you can efficiently configure their back ends, connect them to your app with just a few lines of code, and deploy static web apps in only three steps. Historically, because of their performance issues, managing images and videos is a daunting challenge for developers. Even though you can easily load media to an S3 bucket with AWS Amplify, transforming, compressing, and responsively delivering them is labor intensive and time consuming.

Read more
Cloudinary Helps Move James Hardie’s Experience Online

While COVID has affected most businesses, it has been particularly hard on those that sell products for the physical ‘brick and mortar’ world. One company that literally fits that bill is our Australian customer James Hardie, the largest global manufacturer of fibre cement products used in both domestic and commercial construction. These are materials that its buyers ideally want to see up close, in detail. When customers have questions, they expect personal service.

Read more
How to Build an Enhanced Gravatar Service, Part 2

Part 1 of this post defines the capabilities of an enhanced Gravatar service, which I named Clavatar, and describes the following initial steps for building it:

This post, part 2 of the series, explains how to make Clavatar work like Gravatar and to develop Clavatar’s capabilities of enabling requests for various versions of the images related to user accounts.

Read more
How to Build an Enhanced Gravatar Service, Part 1

The advent of web development since 25 years ago has given birth to an abundance of online services that help developers build apps efficiently and smoothly. Gravatar is a shining example of such a service. Built by WordPress, Gravatar generates globally recognized avatars. Fun fact: 80 percent of Gravatar users first came across that service when reading a WordPress blog.

Read more