Exploring Cloudinary’s Generative Replace: Generative Object Replacement With AI

Cloudinary has added another AI-driven image manipulation feature to Programmable Media: Generative Replace. 

Currently in beta, Generative Replace harnesses the capabilities of natural language prompts and generative AI to detect and replace objects in images. The best part? It does this through Programmable Media’s powerful transformation API. 

With potential applications in content moderation, photography touch-ups, and color removal, Generative Replace adds another dimension to Cloudinary’s suite of generative features that includes Generative Fill and Generative Remove. Like these other two tools, Generative Replace drastically reduces the time spent on tedious and labor-intensive image editing tasks at scale.

Try out Generative Replace and other cutting edge features at our new Generative AI Playground.

Like Generative Remove and Fill, Generative Replace utilizes open-set object detection and diffusion models to identify objects within images. However, instead of simply removing the detected object and inpainting the surrounding area, Generative Replace takes it a step further by using generative AI to intelligently fill the cleared space with an object based on a natural language prompt provided by the user. Let’s take a look at some live examples of the tool in action.

Generative Replace is called via the gen_replace or e_gen_replace transformation effect. Start by calling e_gen_replace, then use the to: and from: prompts to choose the object to remove and what to replace it with.

Here we replace the cookies in the original image with a cupcake with simple to/from prompts:

Loading code examples

In this example, we’re using the same to/from technique to alter the clothes worn by this model:

Loading code examples

By default, e_gen_replace will use what space is needed to replace an object. However, you can ask Generative Fill to use the only area removed with a new object. This adds a layer of precision to the feature. Here we’re getting a little creative with our prompts asking the feature to replace a blank space with an “A Van Gogh style painting of a town at night” with “preserve_geometry” set to true. We know it will only use the space occupied by the object in the “from” prompt:

Loading code examples

Here’s a brief video showing some more examples of Generative Replace.

Create an account to get started using Generative Replace yourself.

Generative Replace opens up a world of possibilities in various industries. Here are some use cases:

1. E-commerce/fashion. Use Generative Replace to change the style of clothing on a model.
2. Advertising. Generative Replace can change background objects, allowing for seamless alterations in product settings. For instance, you could replace a flower bouquet in an image with a pumpkin to create a seasonal-themed ad or product shot that captures the target audience’s attention.
3. Content moderation. Content moderation becomes more efficient and effective with Generative Replace. Users can easily replace unwanted objects to ensure appropriateness for different audiences. For example, a beer bottle can be replaced with a juice bottle, transforming an image into a family-friendly version suitable for a website or campaign.
4. Coming soon: generative re-color. Cloudinary is continuously expanding its generative feature set, and an exciting addition on the horizon is Generative Color Fills. This forthcoming feature will enable users to create color variants of products effortlessly. Users can generate multiple color variant images from a single original model shot by selecting an object via a prompt and changing its color. This functionality will prove invaluable for e-commerce businesses seeking to efficiently and scale showcase product options.

Cloudinary’s Generative Replace feature streamlines object replacement through the power of generative AI and natural language prompts. Users can achieve impressive transformations and enhance their creative workflows by seamlessly replacing objects in images. Whether transforming product photography, ensuring content moderation, or creating dynamic color variants, Generative Replace empowers users with advanced capabilities that help improve their workflows.