How to Automatically Remove Photo Backgrounds in Seconds With AI

Written By Jackie Rosenzveig Aug-23-2020 6 Min Read

These days, e-commerce websites are trending clean and sleek, almost always spotlighting high-quality product photos with a flat-white, solid-color, or transparent background. Many practical and well-validated reasons support this choice, making it clear that this is no passing fad. This graphic illustrates a few of them:

Infographic: Why Remove Photo Backgrounds?

How do website or app developers deliver hundreds or thousands of solid- or transparent-background images, including user-generated content? Getting someone to manually edit them with an editing tool definitely doesn’t scale.

Cloudinary has the solution in its AI Background-Removal Add-On. The add-on’s deep-learning and artificial-intelligence algorithms identify the primary subject in each photo’s foreground and then accurately remove its background in mere seconds.

Moments after you upload your photos, …

These . . .	Become These!

Transparent Backgrounds Are Just a Parameter Away

Cloudinary is a robust media-management platform that leverages automation and deep-learning algorithms to simplify and accelerate workflows for website and app developers, from bulk uploads of raw photos to on-the-fly transformations for creating multiple media variants, optimizations, and responsive delivery to all devices.

To apply the AI Background Removal Add-On to images during or after uploading them to your Cloudinary account, add the background_removal parameter set to cloudinary_ai in your upload or update command. When triggered in the method call, the add-on analyzes the image and then replaces the originally uploaded image with a transparent PNG containing only the foreground image. Cloudinary backs up the original for later retrieval, if required.For example, to have Cloudinary replace the the baseball JPG image above with a transparent PNG, you could run the following upload command:

Even though background removal takes only a few seconds, if you plan to deliver the transparent image on your site or app immediately after upload, it’s a good idea to include the notification_url parameter in your upload or upload command, which activates a webhook that receives the removal status. You can then check if removal is still in progress and, if so, you might want to display a placeholder image or an in-progress animation until the process is complete.

We Only Make It Look Simple

Segmenting objects from images is a fundamental and well-studied challenge in the computer industry. Nowadays, neural networks are the “secret sauce” behind many segmentation tasks that enable state-of-the-art functionality for various applications, such as autonomous driving, medical analysis, and industrial planning.

As shown above, erasing image backgrounds with Cloudinary’s add-on is simple for the developers who use it.. Behind the scenes, however, the Cloudinary engine must perform these three complex tasks::

Identify the salient objects in the image to determine which objects to classify as foreground according to the context and scene composition.
Segment the salient objects. Most neutral networks today produce only coarse maps through standard segmentation techniques. For images slated for production, the segmentation maps of foreground versus background pixels must be near-perfect to achieve seamless background removal, e.g., proper segmentation of fur and hair requires meticulous processing.

Currently, no large public dataset exists for accurate background removal, rendering it all the more difficult to acquire the quantity of media assets necessary for training neutral networks.
Separate the foreground as an alpha layer.

While addressing those challenges, we at Cloudinary also focused on finding ways to optimize our algorithm to deliver the high performance our customers expect; that means processing images of all sizes at scale.

Optimal Results Are Paramount

When you prepare the delivery URLs for the resulting transparent PNGs, you might want to do one or both of the following:

Add a solid or neutral background to the images and then optimize the updated images using Cloudinary’s auto‑format (f_auto) and auto-quality (q_auto) parameters to reduce file sizes with no loss in visual quality.
Eliminate unnecessary white space in the transparent images, which still retain the original dimensions, and then center them within the updated dimensions To do that, just add the auto-cropping parameter, which tells Cloudinary to identify the visible items in the images and then resize and crop them around these foreground objects to the size and aspect ratio you request.

For example, the Background Removal Add-On returns this cute stuffed sheep without the table and wall in the background but with a lot of redundant white space on the right. Also, this transparent PNG boasts a hefty weight of slightly over 1 MB.

How about transforming the image further in the following ways?

Convert the image to a square JPG with a white background.
Optimize the image with f_auto and q_auto.
Add the g_auto:subject parameter to ensure that, no matter the aspect ratio, Cloudinary automatically selects the location for the cropping based on the main—in this case, the only—subject in the image.
Add a thin gray border.

Loading code examples

That way, even if you retain the image’s original height of 1,280 pixels, the delivered version would still weigh only 92 KB.

Another excellent way to eliminate excessive whitespace from images is to apply Cloudinary’s trimming effect (trim), like this:

Loading code examples

Not Everything Is Black and White

Besides those steps performed by the Background Removal Add-On, optimizing images for delivery often involves other transformations, such as replacing the transparent background and converting the image to a non-PNG format. You might wonder why we don’t just return a (smaller) JPG that already has a solid white background.

We’re glad you asked. .

Neutral Colors

In addition to offering the same advantages as those from a white background, a light, neutral-colored background breaks the monotony of an all-white page, sometimes even conveying a certain style or personality without distracting visitors from the main theme.. Walmart.com’s apparel landing-page is one example.

The Walmart Example

To have Cloudinary automatically change the background color, set the value of the background (b_) parameter in your URL to the desired color. For example, here’s the image of the sheep with a very light-blue background.

True Colors

Conversely, some retailers, such as Zara.com, opt for brightly colored backgrounds for white or black products and neutral backgrounds for other merchandise.

The Zara Example

Without a Shadow of a Doubt

A fully transparent background makes it easy to add shadows. Even if the object in the original photograph has a shadow, the Background Removal Add-On will remove that shadow, enabling you to add a consistent shadow location and size for all your product images.

For example:

Remote Control (Original)

Backgroundless Remote Control

Backgroundless remote control with an added shadow

Below is the transformation used for the far right image shown above: :

Loading code examples

Green Screening Without a Green Screen

For fun, add the underlay parameter set to an image’s public ID to add background scenery for a green-screen-like effect..

For example, with an instantaneous on-the-fly transformation, we can send our baseball player to outer space, get the dog off the couch, and place the hiking boots against natural scenery:

Below is the delivery code for the hiking boots image.
(Click the other transformed images in the right column above to see their delivery URLs in your browser.)

Loading code examples

Put It All Together

To display product photos on your own site or marketplaces, or to offer background-removal service for the photos uploaded by your users, you need an automated, efficient, and reliable solution that ultimately delivers images with a sharp and clear foreground image. Once the original background is “out of the picture,” you can freely deliver your product images with white backgrounds, neutral colors, bright colors, or in context of a realistic scene.

Do all of the above if you wish. One way is to place all your background-free images—along with other product images or videos—in a single carousel-controlled gallery, such as Cloudinary’s Product Gallery. For details, see this blog post, this video tutorial, and the Product Gallery documentation.