Rich media is the most effective tool for stimulating customer interest and engagement online. As the volume of digital media grows, much of it ends up in data lakes filled with a huge amount of social media. Such user-generated content (UGC) offers a massive pool of advertising that shoppers find almost as trustworthy as personal recommendations and more engaging than packaged content. Processing that vast, untapped trove of media is humanly impossible, however.
Given the substantial growth of the capabilities of machine-learning tools in the past decade, media analysis and object detection have made their way into many popular applications and environments:
- Health and wellness. AI assistants can support clinicians with expert image analysis, the result of which serves, for example, as a basis for checking crops for diseases and pests.
- Fashion and beauty. AI can help consumers find an outlet to purchase the shoes they see a model wearing in a magazine.
- Art and culture. Museum and gallery patrons can read more about an artifact or installation on an AR application on smartphones.
- Content accessibility. AI is in wide use for generating captions for live video and audio, rendering digital content more accessible.
- Automotive. Many motor-industry leaders combine data from suites of sensors and AI to navigate self-driving cars.
One of the largest applications of AI is unearthing the untapped potential of unbranded media.
The majority of online discussion about brands—that massive volume of UGC—takes place in social-media posts that carry no direct mentions of the brands themselves. Media intelligence can locate opportunities from social media by doing the following:
- Identify authentic responses. You can’t rely on mentions alone because users are notoriously inconsistent in the metadata they attach to their posts, and the limited insight you get misses most of the discussion and hides the real ROI of your marketing campaigns. Media intelligence fills in the details by adding the missing “visual mentions” in unbranded posts to your pool of content for image analysis. You’re then privy to the entire conversation.
- Help develop new media strategies. The media-related details serve as the basis for building new digital experiences.
- Create content networks. Also revealed is a wealth of unmentioned connections your customers usually aren’t aware of. Are you missing out on a world of drone racing if you advertise GoPro accessories to snowboarders only? Should you target gamers with LED lighting, or does it only show up as the silent hero in architectural posts of upscale apartments?
Equipped with a comprehensive picture of where your product sits in the cultural discussion, you can intelligently change branding formats, form new partnerships, and fine-tune your marketing messaging to align with the real user base—not just the people who are the most vocal about your products.
To effectively bring to light new marketing opportunities in the existing content, you must rely on several strengths of media intelligence, as described below. By locating images and videos that contain your brands or content you would like to use, you can create easy purchase opportunities that are tied directly to the media.
Media intelligence can quickly determine the context of an image by means of image-classification models, e.g., one that differentiates between the images of a person, a table, and a salad in posts by a group of friends about their lunch date downtown. That’s a general model for single objects—essentially what an image-based captcha is designed to determine.
Additionally, image-classification models can filter out posts irrelevant to a brand. An outdoor apparel company, for example, can focus its computing resources on images classified as “outdoors” and discard candidates labeled as “art,” “cat,” or “train.”
What if you want to determine if someone shown in one of the lunch-date posts is wearing sneakers, or if a bottled drink is in the photo of the meal? The object-detection technique yields credible answers.
Object-detection models resolve the contents of an image or video more finely than image classifications by determining the locations and classes of the objects. To highlight the objects, the model places square or rectangular boxes around them and then labels them, typically to some level of confidence.For example, even if the caption of the image below posted on social media mentions a hat brand and nothing else, the Cloudinary AI Content Analysis add-on can fish out the post and determine if the image shows jewelry.
Also, if a post does not fit your brand’s criteria, you can discard that image. For example, since the image below shows no bottled drinks, your purée business hasn’t missed a marketing opportunity with this post even though the dining venue mentioned in the caption sells your product.
Let’s look at a more complex example.
As a central component of the shot below, your camera brand is probably already mentioned in the caption. However, the image is flagged as a potential opportunity in a first pass by Nike, but it’ll take additional processing to verify that this post with the label “shoe” does cite the Nike brand.
A semantic segmentation model creates a different mask for different object types within an image, cropped or not. In fact, in some cases, this model understands the object type better on the entire (uncropped) image, which has more context. Preprocessing by cropping can create a higher resolution with segmentation models.
For the Nike example earlier, you can analyze with a segmentation model only the part of the image that contains the shoe. To facilitate that task, you could use a detection model like the Cloudinary one for human anatomy to discard the irrelevant portions of the image before processing the image-segmentation mask.
After images are classified and labeled, the next task is to find the appropriate content for an image bank. Since each of the detection models described above produces indexable, labeled content, finding opportunities is just a matter of leveraging database design and search algorithms.
In addition, detection models can adapt existing, professionally created content that doesn’t meet your needs, such as by removing backgrounds and isolating a subject for adoption in promotional materials.
Once you’ve selected content you’d like to promote or use, AI comes in handy for processing techniques like chroma-keying, whereby machine learning algorithmically finds keyed objects and replaces the rest of the scene around them.
For example, you can refine a kitchen scene for promotional material with the Cloudinary AI Background Removal add-on to process components you’ve found in an image bank and then place them in the scene.
What about creating a series of posts on audience interactions based on a 3D pop-out template? You can use content-aware AI to adapt UGC images to remove their color backdrops without the tedium of manually performing that task. In fact, you could even automate a pipeline to create finished images and then simply select the best-looking ones.
Delivering all that content if it is already optimized at the time of creation is a snap. To that end, apply machine learning to video for more efficient compression of variable bitrates. Content-aware AI can detect scenes or series of frames with fast movement and encode them at a lower bitrate before encoding them again with less movement
Ultimately, companies managing a high volume of media must optimize their media libraries for numerous presentation formats—a burdensome and time-consuming chore if performed manually.
Turn to Cloudinary’s automation and intelligent workflows to efficiently serve media that scales for visual experiences across all touchpoints. You stand to dramatically reduce costs and offer your customers a more seamless, optimized experience.
Trained by machine learning, Cloudinary Media Intelligence yields a powerful object-detection toolset for fine-tuning media assets. One such tool is the Cloudinary AI Content Analysis add-on, which supports these detection models:
- The Large Vocabulary Instance Segmentation model for a broad palette of general objects.
- The Common Objects in Context model for more accurate detection of 80 common objects.
- Google’s Open Images Dataset for an additional 600 general objects.
- Cloudinary’s fashion model, which is dedicated to clothing. If leveraged with automatic image tagging, the model’s response includes attributes of the item identified, e.g., the material and fastenings.
- Cloudinary’s text model, which determines if text is in the image and, if so, the text location. You can then search for images that contain text blocks with automatic image tagging, or keep only the text or specify a crop without the text with object-aware cropping.
- Cloudinary’s human anatomy-model for identifying parts of the human body.
Through built-in, intelligent automation, Cloudinary efficiently processes large volumes of assets throughout the media-asset lifecycle.
Cloudinary media intelligence is customizable for unique brands, subject matters, or specific task requirements.
- Smart cropping: Content-aware, on-demand image transformations keep the subject in focus at any aspect ratio.
- Smart tagging: Adding tags to rich media based on objects or concepts through automation results in efficient categorization and service.
- AI background removal: Automatically removing backgrounds makes the subject the focus.
- Video previews: Automatically generating clips based on the most appealing sections of a video for hover-over thumbnails or fast-loading videos for your feed makes for a captivating preview.
- Auto-optimization: Serving automatically optimized, high-quality video for all devices and connections spells efficiency and peace of mind.
- Video transcription through AI: A neural network model based on Google’s Cloud Speech API automatically transcribes video through a speech-to-text feature so that the content becomes more accessible and SEO friendly.
In summary, media Intelligence makes it possible to efficiently search through the mountain of UGC on social media and untap the sheer volume of potential in unbranded or miscaptioned posts. That’s a compelling target for business expansion.
Behind the scenes, machine learning-powered methods find products in UGC, and media intelligence enables you to draw connections among products and their context for clues of potential partnerships or markets. You can then further organize and process this content, automatically adapting it to your templates or serving it to all touchpoints.
Ultimately, Cloudinary’s media intelligence streamlines the media-management process, from discovery to categorization, and, finally, to optimized delivery according to consumer needs. Besides being customizable for a wide variety of tasks and subjects, Cloudinary’s media intelligence also powers automation throughout the process. For more details, see the documentation on Cloudinary AI Content Analysis.
