Programmable Media release notes: June 19, 2023

New features

AI-powered image captioning (Beta)

Introducing Cloudinary's AI-Powered Captioning!

Automate image caption generation, boost accessibility, and improve your SEO effortlessly.

Powered by a multi-modal large language model (LLM), this solution goes beyond traditional AI auto-tagging to produce descriptive natural language captions that relate to the relationship between the objects in your image.

You can use the returned caption to display product descriptions, add as alt or title text to your image tags for accessibility, to store in your asset's structured metadata or context field for future use, and more.

Below is an example of an image where the automatically generated caption is displayed below it and also used as the alt and title value of the displayed image.

To use this feature, make sure you've enabled the Cloudinary AI Content Analysis add-on and then include the detection parameter set to "captioning" in your upload call or upload preset when uploading new images, or use the update method to apply it to existing images.

The caption value is returned in the response and can then be referenced and used as needed.

Learn more

Remove image objects with generative AI (Beta)

Cloudinary is excited to announce the generative remove effect, which utilizes open-set object detection and powerful generative AI capabilities to remove unwanted objects or custom-defined regions.

To remove an object, use the e_gen_remove transformation and provide natural language prompt text indicating what should be removed from the image. The area of the removed object is automatically repainted based on the image contents, so it's as if it was never there.

The generative remove effect enables creative and technical teams to streamline their workflows and achieve efficient object removal at scale.

Learn more

Example: To remove the stick from the dog's mouth you can use: (e_gen_remove:prompt_the stick):

Original image Remove the stick

New React Native SDK (Beta)

Cloudinary's new React Native library provides image rendering capabilities and plugins that you can implement in your React Native apps including a dedicated image component for easy integration with Cloudinary.

The React Native SDK uses the same url-gen library as the React SDK to generate transformations, making onboarding smoother, and enabling you to more easily deliver Cloudinary optimization and transformation capabilities in your React Native mobile apps.

Learn more

Cacheable Search URLs

You can now use search API expressions to generate a cacheable search URL. You can then deliver that URL from any browser or front-end app to generate a complete search API JSON response. Your URL can be given a specific time to live (300 seconds by default), and then the results of the search are cached on the CDNs for that amount of time before the search results will next be regenerated. This gives you full control of the performance vs liveliness balance of your search.

Since the URL only generates and caches the search results when delivered, you can also take advantage of these URLs in pure client-side apps to implement a variety of frontend features.

Initial generation of the URL requires a signature. You can generate it using the Cloudinary CLI) and then just add the returned URL to a client-side app without the need for any live backend. Alternatively, you can generate the signed URL on the backend using a backend SDK.

For example, the following Python SDK request on the demo account:

...returns the URL below.

You can open this URL in a new tab to view the cached results of the search query:

Learn more

New webhook management API and UI

You can now create, update and delete multiple webhook URL endpoints and control which event types will automatically trigger notifications to each endpoint (without the need to pass the URLs in the event calls themselves).

For example, you could define one URL for all upload events, a different one for events that modify existing assets (explicit, update, tag & metadata changes, etc), and another just to monitor eager transformation events.

You can manage your webhook URLs and select which event(s) trigger each one programmatically with the new /triggers endpoint or via the new Webhook Notifications page in the Console Settings.

Configuring multiple webhook endpoints can be helpful for integrating Cloudinary into existing workflows more seamlessly and makes it easier to trigger flows or synchronize multiple parts of your ecosystem when integrating Cloudinary with other platforms.

It can also help distribute the load and improve the reliability of notification delivery to the desired destinations without overloading a single endpoint, which might result in performance issues.

Learn more: Admin API /triggers endpoint | Webhook Notification Settings

Enhancements

Sort Search API results by relevancy score

By default, the asset list returned in a search method response is sorted in descending order by asset creation date. Until now, you've been able to customize the sort_by order to ascending or descending by any of the asset attribute fields.

Now you can also choose to sort by a relevancy score, so that, for example, query terms that are included in more asset fields, or where more of the query terms exist (in the case of "or" searches), get a higher score.

Frontend SDK examples with imports

One of the great benefits of the Cloudinary frontend SDKs (React, Vue, Angular, JS) is the url-gen library tree shaking, which ensures that the bundle size of your package will be limited to only what your app really needs. This however, results in a need to know exactly which actions and qualifiers you need to import.

All of the transformation examples in the Cloudinary docs as well as the generated transformations in the Transformation Builder now enable you to see and copy the import statements you need for each transformation.

Here's a live example:

Bonus credits when you join the Cloudinary Community

You can now earn 5 free credits for your account by joining the Cloudinary Community. Make sure you log into the Cloudinary Community using the same email, google, or github account that you use to log into the Console.

While there, you'll be able to post questions and get answers from fellow developers and the Cloudinary team, find or share cool tips, learn about new features, and help others solve their media management and delivery challenges.

Register for notifications

Make sure you always know when new release notes are published:

• Programmable Media release notes RSS feed: Grab this RSS link to watch for new Programmable Media release notes in your favorite RSS reader.

• Cloudinary Discord: Join the Cloudinary Discord server and keep an eye on the #cloudinary-news channel. Our RSS feeds will automatically be pushed there whenever new release notes are published.