Skip to content

Expanding Images With Cloudinary’s Generative Fill: AI-Powered Outpainting

The latest Generative AI feature added to the Programmable Media product, Generative Fill, is here. 

At its core, this technology taps into advanced AI algorithms to predict and generate pixels that extend beyond an image’s original borders. It’s more than just stretching and filling out your photos; it’s understanding the context of your original picture and crafting pixels that match its style and setting. This means that the transition looks natural, whether you’re converting a portrait to a landscape or adding more sky to your beach sunset photo.

Building on the Generative Remove and Generative Replace capabilities, Generative Fill also leverages AI to expand and extend original images. This feature is especially valuable when transforming images from vertical to horizontal orientations by seamlessly blending the new AI-generated background with the existing content, which can satisfy many practical use cases.

Like all of the features of Cloudinary’s broad AI offerings, Generative Fill allows you to achieve creative possibilities programmatically, significantly, and at scale, reducing workflow time and increasing your content velocity. If you have multiple images or would like to programmatically adapt generative fill to your website with AI, this feature has been built for you!

Generative Fill is perfect for those who need their images to fit different formats or for anyone looking to add a bit of magic to their visuals. It’s designed to keep the storytelling intact while making your images ready for any platform or layout.

Try out Generative Fill and other cutting-edge features at our new Generative AI Playground.

Check out this video in order to see how generative AI works with real examples. Cloudinary’s API uses a process called outpainting that enables the expansion of images by only modifying the URL.

generative fill

Sign up for free and try it yourself!

Generative Fill works with Cloudinary’s padding crop modes and leverages the new gen_fill option for backgrounds (or b_gen_fill for the URL API). Combining these options allows users to achieve a visually pleasing fill that seamlessly matches the original image. 

Moreover, Generative Fill allows users to add a natural language prompt to the gen_fill command, enhancing creative control over the generated background.

The padding crop modes that work with generative fill backgrounds are:

Pad. Resizes asset with optional padding, maintaining aspect ratio and visibility.

Limit Pad. Scales down an asset with padding if larger than the specified limit, maintaining aspect ratio and visibility.

Minimum Pad. Adds padding to a smaller image to achieve a minimum width/height without scaling, maintaining the aspect ratio of the original.

Fill Pad. Prevents “bad crop” by using fill mode and adding padding if needed, must be used in conjunction with g_auto.


For more information, check out the documentation or try the demo.

Below are some real examples of Generative Fill using our various padding crop modes. We’ll be using the same portrait/vertical image of a  model with a width of 600 and a height of 900 to illustrate the differences between the modes when b_gen_fill is used:

Original image, 600×900

Resizes assets with optional padding, maintaining aspect ratio and visibility of the original with the new areas generatively filled in. For this example, we’ve padded the original to a 1:1 aspect ratio:

Loading code examples

Scales down an asset with padding if larger than the specified width/height limit, maintaining aspect ratio and visibility. The new areas will be generated around the scaled image as needed. Here we’ve created a 400×800 image from the original – the original is shrunk to fit, and new imagery is inserted, including feet!

Loading code examples

Adds padding to an image if the image’s width or height is smaller than the specified dimensions than the original to achieve a minimum width/height without scaling, maintaining the aspect ratio of the original. Here, we’ve maintained the height of the original (900px) but increased the width to 1100px. The new areas to the left and right of the original have been created using Generative Fill.

Loading code examples

Used with g_auto, this mode will attempt to prevent a pad crop by considering auto gravity and applying some padding as needed. Here, we’ve created a unique side banner crop of the image 500*1200 and applied g_auto. As you can see, the focus is on the woman and the trees; b_gen_fill adds lovely matching imagery to the top and bottom of the image to complete the crop.

Loading code examples

For perspective, take a look at this short video which flows through the above examples to highlight how generative fill works with each of these padding modes:

Generative Fill harnesses the power of Cloudinary’s powerful generative AI technology, which combines diffusion models with the platform’s versatile cropping capabilities. 

Generative Fill uses a process known as outpainting, which enables the expansion of images by adding content around the edges. This content smoothly blends with the existing picture, preserving the style and details of the original, resulting in a coherent and extended image.

When a user provides a natural language prompt, the AI considers this and will generate extending pixels that match the original and take the instructions into account.

  1. More appealing crops and padding. Users may include blank padding vertically or horizontally when an image is cropped to a specific aspect ratio. Generative Fill fills these blank pixels intelligently, resulting in a visually pleasing result. Whether creating product images for e-commerce platforms or fitting images into various design layouts, Generative Fill ensures a cohesive and visually engaging presentation.
  2. E-commerce. Extend product images horizontally or vertically to fit the dimensions of product detail or listing pages seamlessly. This transformation ensures that the products are displayed consistently across different devices and platforms. Generative Fill helps enhance the visual appeal of product catalogs and detail pages. Combined with Cloudinary’s powerful optimization features, this can improve the overall shopping experience and, most importantly, conversion rates.
  3. Travel and hospitality. Create stunning wide banner images by extending the dimensions horizontally. Generative Fill enables the creation of captivating hero images that draw users’ attention and showcase the beauty of travel destinations and resorts. 
  4. Use with other Generative features. Combining Generative Fill with Cloudinary’s other generative features like Generative Replace and Generative Remove increases the creative possibilities. These features work harmoniously, enabling users to craft the perfect image from a near-perfect original. Whether removing unwanted objects, replacing elements, or extending the image background, the mix of generative features unlocks limitless creative possibilities through our powerful Image and Video platform.

Cloudinary’s Generative Fill is a powerful feature that expands the possibilities of image manipulation and creativity. By utilizing the power of generative AI technology, users can seamlessly extend and develop images, creating visually appealing results that preserve the style and details of the original content. Whether for e-commerce, travel, or other industries, Generative Fill empowers users to take their visual storytelling to the next level through Programmable Media. With Cloudinary’s suite of generative features, you can achieve automated creative possibilities that seemed impossible a short time ago.

If you are interested in the feature generative fill, you can create your own account for free and test this feature out.

Can I create montage images using Generative Fill with AI?

Yes, you can create montage images with Generative Fill. This feature allows you to blend multiple images into a single composition by filling in and expanding the background areas. This is particularly useful for creating thematic or storytelling visuals where continuity and aesthetic unity are key.

How does AI ensure that the generated parts of an image match the original style?

AI models used for Generative Fill are trained on vast datasets of images, allowing them to learn and replicate various artistic styles and visual elements. When extending an image, the AI analyzes the original content’s context, colors, and textures, ensuring that the new sections are generated in a way that is consistent with the original image.

What are the limitations of AI in Generative Fill?

While AI in Generative Fill can produce impressive results, it has limitations, such as difficulty in accurately replicating extremely detailed or complex patterns found in the original image. Additionally, the AI might struggle with understanding abstract concepts without precise prompts, leading to less predictable outcomes.

How does Generative Fill handle images with multiple focal points?

Generative Fill uses a feature called ‘auto gravity,’ to determine the most important areas of an image. This ensures that when the image is extended, the AI focuses on these key elements, whether they are multiple focal points or a single subject, thereby preserving the image’s narrative integrity.

Is Generative Fill suitable for all types of images?

Generative Fill is particularly effective with images where the background or surrounding context can be logically extended. It is ideal for landscapes, abstract backgrounds, and simpler compositions. For highly detailed or specific imagery, such as close-up portraits or technical diagrams, the results may vary.

How does Generative Fill impact the resolution and quality of the extended image areas?

Generative Fill is designed to maintain the highest possible resolution and quality in the extended areas of an image. The AI algorithms replicate the pixel quality and sharpness of the original image, ensuring that the new sections do not suffer from degradation. However, the ultimate quality can depend on the original image’s resolution and the complexity of the area being filled.

Can Generative Fill be integrated with other AI tools for comprehensive image editing?

Yes, this functionality can be seamlessly integrated with other AI tools provided by Cloudinary and other platforms. This integration allows for a comprehensive suite of image editing options, such as object removal, color correction, and style transfer. Combining these tools can transform an image far beyond simple cropping or resizing, enabling users to fully reimagine their visuals in creative and dynamic ways.
Back to top

Featured Post