Cloudinary Blog

Handling Media at Scale Using Automation and AI

Managing Media at Scale With Automation and AI

In today’s digital world, engaging online content is key for catching audience attention. At ImageCon 2020, Cloudinary’s annual conference, Yaron Reichert and me, Jackie Rosenzveig, hosted a session titled ’Handling Media at Scale Using Automation and AI’, which delved into the following:

  • The importance of creating and maintaining compelling visual content through automation with a focus on four major developer tasks
  • The roles AI and automation play in every stage of the content lifecycle, including enhancements and optimization at scale, for a strong user experience

Below is the video of the full session.

Read on for a quick recap of all the cool AI and automation features our session covers:

Media Asset Lifecycle

In the first part of our presentation, we review the standard lifecycle of an image or a video:

  • Creation: The birth of an image from a photo shoot or graphic design.
  • Editing: Fine-tuning for the purpose of matching a website layout, complying with brand guidelines, and catering to audience preferences.
  • Delivery: Optimization and display that align with the user’s device, network condition, and other factors.
  • Enhancement: Analyses on the degree of success of the content along with insight on areas that need improvement.

AI and automation can play a masterful part in all four of these stages, enabling you to deliver rich and scalable media in a seamless and efficient end-to-end process—with no need for specialized manpower or graphic editing software.

For the remainder of our session, we walk you through four very popular use-cases to show how AI and automation can help speed up and simplify your end-to-end media needs:

Automating Social-Video Creation

With brands relying more and more on social media to reach their target audience, the demand for social videos has skyrocketed. Research consistently shows that videos foster stronger customer engagement and dialog than other content types.

Video Engages and Converts Customers

Even though you can’t predict which social video will be the next to go viral, it's important to keep in mind that all well-crafted and successful videos have a catchy title and are brand consistent. They also invariably apply the appropriate aspect ratio for the social network and contain easy-to-read captions.

In this session, Yaron demonstrates how Cloudinary's sophisticated AI-driven "g_auto" algorithms can automatically crop your videos to fit the various platforms, in particular cropping landscape videos to the required square and portrait aspect ratios, while keeping the main subject in the frame throughout the video!

He also talks about the importance of including titles and captions to capture the attention of the large percentage of your audience who may start watching your videos on mute, the value of a modern progress bar or frame to give users an idea of video progress, and consistent branding in the form of recognizable logos, fonts, and colors. Cloudinary makes it easy to add these programmatically to all of your videos with simple transformation parameters.

Enhancing the reliability and user experience of User-Generated Content (UGC)

Social-media users love viewing one another’s comments, opinions, and the like, but only if those contributions are visually appealing. On the average, industries enjoy a rise of 161 percent in the conversion rate of page visitors who see UGC and those who don’t. However, tremendous room for error exists in UGC, such as design mismatches and noncompliance with the business’s quality or security standards.

In the second part of our session, I got to show how you can "maximize the experience while minimizing the surprises of UGC with, you guessed it -- AI and automation -- including:

  • Evaluate images for quality and then flag (or reject) over-pixelated images
  • Check whether an uploaded image will work well for your color-blind users .
  • Scan for malware and inappropriate content
  • Automatically crop uploaded images around a specific object type or product category
  • Automatically generate image tags for SEO
  • Capture metadata from users
  • Enhance visuals with stylistic filters, cool effects, and other visual adjustments

UGC Slide

Optimizing Videos

Today’s online videos must load quickly, play smoothly, and look impressive; otherwise, visitors will move on.

High-quality videos exact a trade-off between quality, size, speed, and the cost of the compression process. Additionally, different devices and browsers require different formats and codecs. Applying adjustments around bitrate, keyframe intervals and frame rate can all impact the visual quality and bandwidth required to deliver your videos.

Complex video

In part 3 of our session, Yaron points out that attempting to create one video that fits all formats and devices is not the answer because this would require going for the lowest common denominator. On the other hand, as browser formats change and new capabilities emerge, manually formatting videos to accommodate all the platforms and devices would be a nearly impossible task.

AI coupled with automation is the right path to take for the capability to dynamically pinpoint the “sweet spot” for optimal bandwidth, speed, and quality for any video on any platform. Yaron shows how to take advantage of Cloudinary's automatic format and quality selection features (aka f_auto and q_auto) to ensure you are always delivering that perfect balance of visual excellence and great performance to every viewer.

Programmatically Managing Media via CLI

Many businesses maintain a large repository of media assets. Managing them includes operations like criteria-based searches, bulk operations on assets, as well as creating and modifying various media entities that you need for your apps. Even though Cloudinary has offered these capabilities pretty much from the start (via its Admin and Upload APIs), these operations usually required you to have a backend application.

In the last few minutes of our session, I walk you through the 3 quick steps involved in setting up and running commands with Cloudinary’s new command-line interface (CLI). Using it, you can run the same administration and upload operations that our server-side SDKs and REST APIs offer, with simple CLI commands. But wait, there's more! The CLI also comes with several special helper methods to help you do things like migrate external files to Cloudinary, sync files between a local folder and Cloudinary or quickly generate the code for any of our embeddable widgets. Super cool stuff!

CLI slide

Bottom line, the CLI means that programmatically organizing and optimizing your extensive media catalogs can be smooth, reliable, fast, and even easy!

The Power of AI and Automation for Media

Whether you are optimizing your own social content, UGC, or video; or managing media internally within your organization, you are probably already painfully aware of the common pitfalls many businesses face today in working with manual processes, especially as your media needs and your application reach scales.

The many features this ImageCon session demonstrates as it walks through the 4 super-common use-cases we described above, have one thing common. They all involve complex AI and automation algorithms that you can easily activate with single lines of code e, to handle your media needs, regardless of scale.

Cloudinary’s user-oriented offerings were developed through countless hours of research, deep learning, and coding. We’re dedicated to helping you exceed customer expectations, which will continue to rise.

Recent Blog Posts

On-Demand Viewing of Live Video Presents New Opportunities

In early 2020, Cloudinary was planning its fourth annual ImageCon conference, a two-day event in the heart of San Francisco, where we’d congregate with curious digital-media minds to brainstorm best practices for media management. Instead, the COVID-19 pandemic forced the entirety of ImageCon 2020 online. As with all other events being planned, we had to overhaul the content to be communicated on video. Gratifyingly, we found the right partner—the event platform Bizzabo—to turn that into a reality.

Read more
Why the Future of E-commerce Is Live

In a previous post, I discussed how “going live” is gaining popularity across industries and verticals. What began as a way for gamers to jam together has evolved into a medium for broader entertainment and business purposes. To continue the conversation, this post unpacks the current trends of shoppable live streams to shine a light on how brands are leveraging “lives” to connect with shoppers in new ways.

Read more
An Overview of Live-Streaming Video Trends

“Let’s go live.” For decades, that’s what newscasters say as they cut to real-time footage of a colleague reporting in the field. The live-video feed adds visual interest and perspective to a story beyond what can be communicated by someone sitting behind the news desk. In the same way, live-streaming video nowadays adds context to other consumer environments. From gaming and events to shopping and social media, “going live” enhances everyday experiences, and it’s something anyone can do with relative ease.

Read more
Readying Live Streams for Video on Demand

When planning a live broadcast or stream, companies often overlook the redistribution phase, but live-stream videos are useful well beyond their initial streaming. Why? Because not everyone watches the first run. For a wider audience, it makes sense to repost live content on your website under an “events” tab, on YouTube, and other social sites for video on demand (VOD). However, preparing footage for reposting can be a lot of work.

Read more