Programmable Media release notes: October 9, 2024

New features

Cloudinary AI Vision Add-on (Beta)

We invite you to check out the Cloudinary AI Vision add-on (Beta) for advanced prompt-based AI image analysis that you can apply on images in your Cloudinary account or on any remote image URI.

The add-on uses Cloudinary's new Analyze API to apply LLM (Large Language Model) capabilities, specialized models, advanced algorithms, prompt engineering, and more to interpret and respond to visual content queries. This enables you to tailor, streamline, and automate image analysis that aligns with your unique brand requirements and your customers' expectations.

The add-on enables you to perform image analysis in 3 modes:

• Tagging: Returns recommended categorization tags based on natural language criteria you define for each category.
• Moderation: Returns boolean responses to natural language yes/no questions about whether the image contains objects you need to moderate on.
• General: Enables you to get natural language descriptive answers about the image content in response to your questions or requests (e.g. "Describe the image in detail" or "How many people are wearing hats?")

Example AI Vision moderation request and image:

Response:

Video Player Studio transcript editor

Automatic speech recognition and transcription tools are essential components of the video experience, enhancing content accessibility, increasing engagement, and providing a deeper understanding of the content’s sentiment and overall tone. However, automatic transcription is never 100% perfect.

Take advantage of the new transcript editor UI in the Video Player Studio to trigger automatic generation of a transcript for your video. You can then edit the generated transcript, including adding or merging lines and editing specific words.

Learn more about the transcript editor

New Java SDK Major Version

Java SDK v2.0 is a major update that brings improvements in security, performance, and compatibility, including:

• Secure (HTTPS) URLs by default: It's no longer necessary to set secure=true in your SDK configuration to ensure that all assets are delivered via HTTPS, even if embedded in non-secure HTTP pages.
• HTTP client updated: The HTTP client has been upgraded to version 5, improving performance and security.
• Security fixes: Addressed all known vulnerabilities.
• Old Java version support dropped: To enhance performance and security, this version is compatible only with Java 8 and later.
• Smaller bundle size: Reduced SDK size for faster downloads.
• Deprecated code removed: Code related to deprecated features was removed for a cleaner and simplified codebase.

New Android SDK Major Version

Android SDK v3.0 major version has been released to ensure compatibility with the latest tools and dependencies, including:

• Fresco package bumped: Upgraded to the latest version
• Java version: Updated from Java 8 to 11
• Minimum Android SDK support update: The Cloudinary Android SDK now supports Android SDK 21 and later (previously 19+)
• CI pipeline: Tests have been migrated from Travis to GitHub Actions

Enhancements

Alternate audio tracks in video player

When using automatic streaming profiles, you can now define alternate audio tracks that will be added to your generated manifest file. This allows you to enhance accessibility, support multiple languages, or offer unique audio experiences such as descriptive soundtracks.

When delivered using a compatible video player (such as the Cloudinary Video Player), users will be able to select their preferred audio track from the UI.

Try out the original vs instrumental sound tracks using the Audio Track toolbar button in the video player below:

Learn how to define alternate audio tracks in your video players

Video Player Studio in Media Library and Media Library Widget

You can now open the Video Player Studio for a specific video by right-clicking it in the DAM Media Library or from the asset drill-down page. It's similarly available in the Media Library widget, which can be integrated into other CMS, PIM, and ECommerce platforms.

You can also continue to access the Video Player Studio from the Console navigation menu under the Programmable Media product, by selecting Video > Player.

Video analytics custom fields

In addition to the fields we capture automatically for your video analytics, you can also capture up to 5 additional custom fields. This allows you to define, submit, and report on data points specific to your use case, such as page title, category, SKU, or subscription plan type. You can also report on video behaviors such as autoplay, looping, and more. Once your custom metrics have been defined and captured they will appear in your Video Analytics dashboard, allowing you to filter your data based on these metrics.

iOS SDK video preprocess transcoding

We're excited to announce the release of the new video preprocess transcoding features for our iOS SDK, which can simplify uploading large video files, especially in low-connectivity environments. The preprocess options include:

• Set output format
• Add compression factor
• Set minimum and maximum dimensions
• Use the dimensions validator (ensures requested dimensions can be applied)

For example, you could define an upload that scales the video if needed to ensure it will be between 250x250 px and 500x500 px, changes the format to MP4, and applies an Apple AVFoundation compression preset.

Learn how to apply video preprocess transcoding

Improved JPEG encoding for smaller size, higher quality images

Cloudinary has begun using Jpegli, a new JPEG encoder, which generates smaller size, higher quality images in the JPEG format. This encoder is being used to generate derived JPEG assets that apply automatic quality (q_auto), either explicitly in the transformation URL or via the default image quality setting.

Jpegli will be gradually rolled out for all Enterprise and self-serve accounts that are using the image impressions metric. Additional improvements are planned in the following months.

New Code Explorers "Try it" page

The Code Explorers: Try it page provides you with an embedded no-scroll 'mini-app' where you can select your favorite language and the sample functionality you want to load the relevant explorer (sandbox) where you can start playing inline.

We think this will be a fun and quick way for you to onboard to Cloudinary or to specific features you've never tried. Each example also comes with an accompanying GitHub repo you can fork to continue experimenting.

Click the rating stars on the right of the Code Explorers page to let us know what you think!

SDK-specific video tutorial pages

We know that lots of developers prefer to learn by watching rather than reading, and we've always offered lots of video tutorials sprinkled throughout our docs, as well as concentrated and organized by functional areas in our Video Tutorial Library.

But we thought it could be helpful to also give you quick, centralized access to all the video tutorials that are relevant for a specific SDK, as well. That's why we've now added a video tutorials page into each SDK guide.

Announcements

Hacktoberfest is on

Hacktoberfest is an annual month-long celebration for Open Source projects that takes place throughout October and there's still plenty of time to take part!

During Hacktoberfest, Cloudinary is opening up a variety of libraries and SDKs to give you the opportunity to contribute, including the official SDKs, community SDKs, and sample projects.

The official program offers digital badges for all participants. However, if you participate with a valid submission to a Cloudinary project, we’ll also send you a free Cloudinary swag pack, including a T-shirt, the famous Cloudinary unicorn plush, and limited edition stickers.

Get all the details here.

JPEG2000 support deprecation

Following Apple's IOS 18 update that removed support for JPEG2000 (jp2) in the Safari browser, and because it is an older and less efficient format, this format is no longer being returned when using Cloudinary's automatic format selection (f_auto).

If needed, you can still explicitly generate an image in the .jp2 format using f_jp2 or setting jp2 as the extension for a delivery URL.

Register for notifications

Make sure you always know when new release notes are published:

• Programmable Media release notes RSS feed: Grab this RSS link to watch for new Programmable Media release notes in your favorite RSS reader.

• Cloudinary Discord: Join the Cloudinary Discord server and keep an eye on the #cloudinary-news channel. Our RSS feeds will automatically be pushed there whenever new release notes are published.