Cloudinary Blog

How to Automatically Remove Photo Backgrounds in Seconds With AI

Remove Photo Backgrounds Automatically with AI

These days, clean and sleek e-Commerce web sites are the way to go. And that almost always includes high quality product photos with a flat white, transparent, or solid background. This is not just a passing fad. There are many practical and well-validated reasons for this choice. The graphic below summarizes a few of these:

Infographic: Why remove photo backgrounds?

But if you are a website or app developer who needs to deliver hundreds or thousands of images, or if your site allows its users to upload product images to your site, getting someone to manually edit photos in a graphic editing tool just won't cut it.

Today, Cloudinary is proud to introduce it's AI Background Removal Add-on, which combines a variety of deep-learning and artificial intelligence algorithms to recognize the primary foreground subject of photos and then accurately remove photo backgrounds in a matter of seconds.

Moments after you upload your photos...

These:

Become These!

Dog on sand - original

Dog on sand - no background

Baseball player - original

baseball player - no background

Transparent backgrounds are just a parameter away

Cloudinary is a heavy-duty image and video platform that leverages automation and deep learning algorithms to simplify the entire rich-media workflow for websites and apps -- from the bulk upload of raw photo shots to on-the-fly manipulations for creating multiple variants of media files, automatic optimizations, and responsive delivery across various devices.

The AI Background Removal Add-on can be applied to any image either while uploading it to your Cloudinary account or afterwards, simply by adding the background_removal parameter set to cloudinary_ai in your upload or update method.

To instruct Cloudinary to remove the background from the baseball image shown above while uploading, your upload command (using your chosen Cloudinary SDK) would look something like this:

Ruby:
Cloudinary::Uploader.upload("baseball.jpg",
  :public_id => "baseball_no_bg",
  :background_removal => 'cloudinary_ai',
  :notification_url => "https://mysite.example.com/hooks")
PHP:
\Cloudinary\Uploader::upload("baseball.jpg", 
  array(
    "public_id" => "baseball_no_bg",
    "background_removal" => "cloudinary_ai",
    "notification_url" => "https://mysite.example.com/hooks"));
Python:
cloudinary.uploader.upload("baseball.jpg",
  public_id = "baseball_no_bg",
  background_removal = "cloudinary_ai",
  notification_url = "https://mysite.example.com/hooks")
Node.js:
cloudinary.v2.uploader.upload("baseball.jpg", 
  { public_id: "baseball_no_bg",
    background_removal: "cloudinary_ai",
    notification_url: "https://mysite.example.com/hooks" }),
  function(error, result){console.log(result);});
Java:
cloudinary.uploader().upload("baseball.jpg", 
  ObjectUtils.asMap(
    "public_id", "baseball_no_bg",
    "background_removal", "cloudinary_ai",
    "notification_url", "https://mysite.example.com/hooks"));
.Net:
var uploadParams = new ImageUploadParams(){
  File = new FileDescription(@"baseball.jpg"),
  PublicId = "baseball_no_bg",
  BackgroundRemoval = "cloudinary_ai",
  NotificationUrl = "https://mysite.example.com/hooks"};
var uploadResult = cloudinary.Upload(uploadParams);

Initially, your original image is uploaded to Cloudinary, and then the add-on removes the background and replaces the original with a (transparent background) PNG. The original image is automatically backed up and can be easily recovered if needed.

Note the notification_url used in the above upload command. Even though the background removal process only takes a few seconds, if you plan to deliver the transparent image on your site or app right after it uploads, you can use the notification webhook to check the status of the background removal and potentially display a placeholder image or an in-progress animation until the process is complete.

We only make it look simple

Activating the add-on to erase all traces of the background is really as easy as shown above. But what's going on behind the scenes is anything but simple.

Automatically removing a photo background requires the engine to:

  • Recognize the salient object(s) in the image
  • Accurately segment those object/s
  • Separate the foreground to an alpha layer

The ability to segment objects from an image is a fundamental and well-studied challenge in computer vision. Recently, neural networks have been able to achieve the state-of-the-art results that have been used in applications as varied as autonomous driving, medical image analysis, industrial planning and more.

But that's just the tip of the iceberg for image background removal:

First, you need to determine which objects in the image should be classified as foreground versus the background, which depends on the context and composition of the scene.

Additionally, the standard object segmentation used in most neural networks produces coarse segmentation maps. But for production-level background removal, the segmentation maps of foreground vs. background pixels must be near-perfect to make seamless background removal possible. For instance, fur and hair require special care to segment them properly.

Furthermore, there is currently no significantly large public dataset for accurate background removal, which makes it difficult to acquire the quantity of media assets necessary to train a neural network.

And even while addressing all these challenges, we knew we had to find ways to optimize our algorithm so it could deliver the fast results Cloudinary customers expect on images of any size, and at the huge scales that our customer base requires.

Delivering Optimal Results

Once the add-on removes the background, you can then deliver the resulting transparent PNG as-is, or you may want to add a solid or other neutral background and deliver using Cloudinary's auto‑format (f_auto) and auto-quality (q_auto) optimizations to achieve smaller files while retaining visual quality.

Additionally, the returned transparent image retains the original image dimensions. This means your resulting image may have a lot of white space and may not be centered within those original dimensions. To address this, you can also take advantage of our auto-gravity cropping, to automatically detect the visible item(s) in the image no matter where they are, and then resize and crop to your desired delivery size and aspect ratio without cropping off any of the foreground image that Cloudinary's deep learning algorithm just worked so hard to find.

For example, using the background removal add-on, this cute stuffed sheep returns without the table and wall background, but it sure does have a lot of white space on the right side. And as a transparent PNG, it's quite a fat sheep, weighing in at a bit over 1 MB.

stuffed sheep - original stuffed sheep - no background

But let's deliver our resulting (transparent background) image as a square JPG with a white background, while using f_auto and q_auto to deliver in the most optimized format for the requesting browser. We'll also add g_auto:subject to make sure that no matter what aspect ratio we choose for our crop, the location for the crop is automatically selected based on the main (in this case, the only) subject in the image. For demonstration purposes here, we'll also add a thin gray border.

Ruby:
cl_image_tag("docs/rmv_bgd/stuffed.jpg", :transformation=>[
  {:quality=>"auto", :gravity=>"auto:subject", :height=>1280, :aspect_ratio=>"1", :background=>"white", :crop=>"thumb"},
  {:border=>"1px_solid_gray"}
  ])
PHP:
cl_image_tag("docs/rmv_bgd/stuffed.jpg", array("transformation"=>array(
  array("quality"=>"auto", "gravity"=>"auto:subject", "height"=>1280, "aspect_ratio"=>"1", "background"=>"white", "crop"=>"thumb"),
  array("border"=>"1px_solid_gray")
  )))
Python:
CloudinaryImage("docs/rmv_bgd/stuffed.jpg").image(transformation=[
  {'quality': "auto", 'gravity': "auto:subject", 'height': 1280, 'aspect_ratio': "1", 'background': "white", 'crop': "thumb"},
  {'border': "1px_solid_gray"}
  ])
Node.js:
cloudinary.image("docs/rmv_bgd/stuffed.jpg", {transformation: [
  {quality: "auto", gravity: "auto:subject", height: 1280, aspect_ratio: "1", background: "white", crop: "thumb"},
  {border: "1px_solid_gray"}
  ]})
Java:
cloudinary.url().transformation(new Transformation()
  .quality("auto").gravity("auto:subject").height(1280).aspectRatio("1").background("white").crop("thumb").chain()
  .border("1px_solid_gray")).imageTag("docs/rmv_bgd/stuffed.jpg");
JS:
cloudinary.imageTag('docs/rmv_bgd/stuffed.jpg', {transformation: [
  {quality: "auto", gravity: "auto:subject", height: 1280, aspectRatio: "1", background: "white", crop: "thumb"},
  {border: "1px_solid_gray"}
  ]}).toHtml();
jQuery:
$.cloudinary.image("docs/rmv_bgd/stuffed.jpg", {transformation: [
  {quality: "auto", gravity: "auto:subject", height: 1280, aspect_ratio: "1", background: "white", crop: "thumb"},
  {border: "1px_solid_gray"}
  ]})
React:
<Image publicId="docs/rmv_bgd/stuffed.jpg" >
  <Transformation quality="auto" gravity="auto:subject" height="1280" aspectRatio="1" background="white" crop="thumb" />
  <Transformation border="1px_solid_gray" />
</Image>
Angular:
<cl-image public-id="docs/rmv_bgd/stuffed.jpg" >
  <cl-transformation quality="auto" gravity="auto:subject" height="1280" aspect-ratio="1" background="white" crop="thumb">
  </cl-transformation>
  <cl-transformation border="1px_solid_gray">
  </cl-transformation>
</cl-image>
.Net:
cloudinary.Api.UrlImgUp.Transform(new Transformation()
  .Quality("auto").Gravity("auto:subject").Height(1280).AspectRatio("1").Background("white").Crop("thumb").Chain()
  .Border("1px_solid_gray")).BuildImageTag("docs/rmv_bgd/stuffed.jpg")
Android:
MediaManager.get().url().transformation(new Transformation()
  .quality("auto").gravity("auto:subject").height(1280).aspectRatio("1").background("white").crop("thumb").chain()
  .border("1px_solid_gray")).generate("docs/rmv_bgd/stuffed.jpg");
iOS:
imageView.cldSetImage(cloudinary.createUrl().setTransformation(CLDTransformation()
  .setQuality("auto").setGravity("auto:subject").setHeight(1280).setAspectRatio("1").setBackground("white").setCrop("thumb").chain()
  .setBorder("1px_solid_gray")).generate("docs/rmv_bgd/stuffed.jpg")!, cloudinary: cloudinary)
Sheep JPG with white background

Now we get our sheep with a solid-white background, and even if you were to retain the original 1280 px height, the delivered image would still only weigh 92kb.

Another great way you can deliver your image with no excess whitespace is to apply Cloudinary's trim transformation effect, which trims off all excess background:

Ruby:
cl_image_tag("docs/rmv_bgd/stuffed.jpg", :transformation=>[
  {:quality=>"auto", :effect=>"trim", :background=>"white"},
  {:border=>"1px_solid_gray"}
  ])
PHP:
cl_image_tag("docs/rmv_bgd/stuffed.jpg", array("transformation"=>array(
  array("quality"=>"auto", "effect"=>"trim", "background"=>"white"),
  array("border"=>"1px_solid_gray")
  )))
Python:
CloudinaryImage("docs/rmv_bgd/stuffed.jpg").image(transformation=[
  {'quality': "auto", 'effect': "trim", 'background': "white"},
  {'border': "1px_solid_gray"}
  ])
Node.js:
cloudinary.image("docs/rmv_bgd/stuffed.jpg", {transformation: [
  {quality: "auto", effect: "trim", background: "white"},
  {border: "1px_solid_gray"}
  ]})
Java:
cloudinary.url().transformation(new Transformation()
  .quality("auto").effect("trim").background("white").chain()
  .border("1px_solid_gray")).imageTag("docs/rmv_bgd/stuffed.jpg");
JS:
cloudinary.imageTag('docs/rmv_bgd/stuffed.jpg', {transformation: [
  {quality: "auto", effect: "trim", background: "white"},
  {border: "1px_solid_gray"}
  ]}).toHtml();
jQuery:
$.cloudinary.image("docs/rmv_bgd/stuffed.jpg", {transformation: [
  {quality: "auto", effect: "trim", background: "white"},
  {border: "1px_solid_gray"}
  ]})
React:
<Image publicId="docs/rmv_bgd/stuffed.jpg" >
  <Transformation quality="auto" effect="trim" background="white" />
  <Transformation border="1px_solid_gray" />
</Image>
Angular:
<cl-image public-id="docs/rmv_bgd/stuffed.jpg" >
  <cl-transformation quality="auto" effect="trim" background="white">
  </cl-transformation>
  <cl-transformation border="1px_solid_gray">
  </cl-transformation>
</cl-image>
.Net:
cloudinary.Api.UrlImgUp.Transform(new Transformation()
  .Quality("auto").Effect("trim").Background("white").Chain()
  .Border("1px_solid_gray")).BuildImageTag("docs/rmv_bgd/stuffed.jpg")
Android:
MediaManager.get().url().transformation(new Transformation()
  .quality("auto").effect("trim").background("white").chain()
  .border("1px_solid_gray")).generate("docs/rmv_bgd/stuffed.jpg");
iOS:
imageView.cldSetImage(cloudinary.createUrl().setTransformation(CLDTransformation()
  .setQuality("auto").setEffect("trim").setBackground("white").chain()
  .setBorder("1px_solid_gray")).generate("docs/rmv_bgd/stuffed.jpg")!, cloudinary: cloudinary)
Sheep JPG with white background

Everything isn't always black and white

As you can see, optimized image delivery will often involve replacing the transparent background and delivering in a format other than the PNG that our background removal add-on initially returns. But there are good reasons why we don't just return a solid white JPG to begin with.

Going neutral

For starters, delivering with a light, neutral background color can provide the same advantages as white, but it can break the monotony of an all-white page, and even communicate a certain style or personality without taking the focus away from the product. In our research, we've seen a variety of retailers using that strategy. For example, here's the approach Walmart.com has taken on their apparel landing page:

Walmart example

To accomplish something similar with Cloudinary, just change the value of the background (b_) parameter in your URL to the desired color. For example, check out our sheep with a very light blue background.

True colors

Conversely, we've seen some retailers going for brightly colored backgrounds when presenting white or black products, while sticking with neutral backgrounds with their other products, like this example from Zara.com:

Zara example

Without a shadow of a doubt

Starting with a fully transparent background also makes it easy to add shadows. Of-course you may have already had some sort of shadow in the original photograph, but our Background Removal add-on removes that shadow, enabling you to add a consistent shadow location and size for all your products.

For example:

Original remote control image Original remote control image  No background No background No background + added shadow No background + added shadow

Here's the transformation we used above to add a shadow to the background-less remote control:

Ruby:
cl_image_tag("docs/rmv_bgd/remote.png", :transformation=>[
  {:width=>200, :crop=>"scale"},
  {:effect=>"shadow:50", :x=>6, :y=>6}
  ])
PHP:
cl_image_tag("docs/rmv_bgd/remote.png", array("transformation"=>array(
  array("width"=>200, "crop"=>"scale"),
  array("effect"=>"shadow:50", "x"=>6, "y"=>6)
  )))
Python:
CloudinaryImage("docs/rmv_bgd/remote.png").image(transformation=[
  {'width': 200, 'crop': "scale"},
  {'effect': "shadow:50", 'x': 6, 'y': 6}
  ])
Node.js:
cloudinary.image("docs/rmv_bgd/remote.png", {transformation: [
  {width: 200, crop: "scale"},
  {effect: "shadow:50", x: 6, y: 6}
  ]})
Java:
cloudinary.url().transformation(new Transformation()
  .width(200).crop("scale").chain()
  .effect("shadow:50").x(6).y(6)).imageTag("docs/rmv_bgd/remote.png");
JS:
cloudinary.imageTag('docs/rmv_bgd/remote.png', {transformation: [
  {width: 200, crop: "scale"},
  {effect: "shadow:50", x: 6, y: 6}
  ]}).toHtml();
jQuery:
$.cloudinary.image("docs/rmv_bgd/remote.png", {transformation: [
  {width: 200, crop: "scale"},
  {effect: "shadow:50", x: 6, y: 6}
  ]})
React:
<Image publicId="docs/rmv_bgd/remote.png" >
  <Transformation width="200" crop="scale" />
  <Transformation effect="shadow:50" x="6" y="6" />
</Image>
Angular:
<cl-image public-id="docs/rmv_bgd/remote.png" >
  <cl-transformation width="200" crop="scale">
  </cl-transformation>
  <cl-transformation effect="shadow:50" x="6" y="6">
  </cl-transformation>
</cl-image>
.Net:
cloudinary.Api.UrlImgUp.Transform(new Transformation()
  .Width(200).Crop("scale").Chain()
  .Effect("shadow:50").X(6).Y(6)).BuildImageTag("docs/rmv_bgd/remote.png")
Android:
MediaManager.get().url().transformation(new Transformation()
  .width(200).crop("scale").chain()
  .effect("shadow:50").x(6).y(6)).generate("docs/rmv_bgd/remote.png");
iOS:
imageView.cldSetImage(cloudinary.createUrl().setTransformation(CLDTransformation()
  .setWidth(200).setCrop("scale").chain()
  .setEffect("shadow:50").setX(6).setY(6)).generate("docs/rmv_bgd/remote.png")!, cloudinary: cloudinary)

Green-screening without a green screen

If you want to have some fun (or pass that fun on to your users), you can add a fun background scene behind your transparent image for a green-screen-like effect, regardless of the original background. Just use the underlay parameter set to the public ID of any image in your Cloudinary account as the new background layer.

For example, with an instantaneous on-the-fly transformation, we could send our baseball player to outer space, get the dog off the couch, or place hiking boots product in a more nature-oriented environment:

Baseball player - original photo Baseball player in outer space

Lazy dog - original photo Lazy dog at the beach

Hiking boots - original photo Hiking boots in nature

Below is the delivery code for the hiking boots example.
(Click the other transformed images on the right above to view their delivery URLs.)

Ruby:
cl_image_tag("docs/rmv_bgd/shoes.png", :transformation=>[
  {:width=>800, :quality=>"auto", :crop=>"scale"},
  {:underlay=>"docs:canyon2", :gravity=>"south", :height=>1000}
  ])
PHP:
cl_image_tag("docs/rmv_bgd/shoes.png", array("transformation"=>array(
  array("width"=>800, "quality"=>"auto", "crop"=>"scale"),
  array("underlay"=>"docs:canyon2", "gravity"=>"south", "height"=>1000)
  )))
Python:
CloudinaryImage("docs/rmv_bgd/shoes.png").image(transformation=[
  {'width': 800, 'quality': "auto", 'crop': "scale"},
  {'underlay': "docs:canyon2", 'gravity': "south", 'height': 1000}
  ])
Node.js:
cloudinary.image("docs/rmv_bgd/shoes.png", {transformation: [
  {width: 800, quality: "auto", crop: "scale"},
  {underlay: "docs:canyon2", gravity: "south", height: 1000}
  ]})
Java:
cloudinary.url().transformation(new Transformation()
  .width(800).quality("auto").crop("scale").chain()
  .underlay(new Layer().publicId("docs:canyon2")).gravity("south").height(1000)).imageTag("docs/rmv_bgd/shoes.png");
JS:
cloudinary.imageTag('docs/rmv_bgd/shoes.png', {transformation: [
  {width: 800, quality: "auto", crop: "scale"},
  {underlay: new cloudinary.Layer().publicId("docs:canyon2"), gravity: "south", height: 1000}
  ]}).toHtml();
jQuery:
$.cloudinary.image("docs/rmv_bgd/shoes.png", {transformation: [
  {width: 800, quality: "auto", crop: "scale"},
  {underlay: new cloudinary.Layer().publicId("docs:canyon2"), gravity: "south", height: 1000}
  ]})
React:
<Image publicId="docs/rmv_bgd/shoes.png" >
  <Transformation width="800" quality="auto" crop="scale" />
  <Transformation underlay="docs:canyon2" gravity="south" height="1000" />
</Image>
Angular:
<cl-image public-id="docs/rmv_bgd/shoes.png" >
  <cl-transformation width="800" quality="auto" crop="scale">
  </cl-transformation>
  <cl-transformation underlay="docs:canyon2" gravity="south" height="1000">
  </cl-transformation>
</cl-image>
.Net:
cloudinary.Api.UrlImgUp.Transform(new Transformation()
  .Width(800).Quality("auto").Crop("scale").Chain()
  .Underlay(new Layer().PublicId("docs:canyon2")).Gravity("south").Height(1000)).BuildImageTag("docs/rmv_bgd/shoes.png")
Android:
MediaManager.get().url().transformation(new Transformation()
  .width(800).quality("auto").crop("scale").chain()
  .underlay(new Layer().publicId("docs:canyon2")).gravity("south").height(1000)).generate("docs/rmv_bgd/shoes.png");
iOS:
imageView.cldSetImage(cloudinary.createUrl().setTransformation(CLDTransformation()
  .setWidth(800).setQuality("auto").setCrop("scale").chain()
  .setUnderlay("docs:canyon2").setGravity("south").setHeight(1000)).generate("docs/rmv_bgd/shoes.png")!, cloudinary: cloudinary)

Putting it all together

Whether you are delivering product photos on your own site or on multiple marketplaces, or you want to offer background removal services for your end users' uploaded photos, you need an automated solution that will quickly and reliably remove photo backgrounds, leaving you with a sharp and clear foreground image.

Once the original background is 'out of the picture', you can freely deliver your product images with white backgrounds, neutral or even bright colors, or you can put them in context of a realistic scene.

In fact, you may want to do all of the above. One way to do this is to put all your resulting background-free images together, along with other views such as 3D images and videos, in a single carousel-controlled gallery, using Cloudinary's easy-to-implement Product Gallery. You can learn more in the Product Gallery blog post, video tutorial, and documentation.

Your turn!

To get started removing backgrounds yourself, just sign up for a free Cloudinary account, register for the free trial tier of our Cloudinary AI Background Removal add-on, and check out the Background Removal add-on documentation.

We'd love to see your background removal photo results in the comments!

Recent Blog Posts

An Eye-Opening Talk: Building Apps for the Next Billion Users in Africa

William (iChuloo) Imoh, who hails from Lagos, Nigeria, recently embarked on a U.S. speaking tour, February 20-March 12, during which he powwowed with technical and product teams and communities at such renowned enterprises as Netlify, Pluralsight, Lucidchart, Twilio, and more in Salt Lake City, Dallas, Las Vegas, and San Francisco. On March 5, he gave an enlightening talk, entitled International Developers and Development: Building for the Next Billion Users at Cloudinary in Santa Clara, California. Below is a synopsis. For details, see the related slides.

Read more
The Debut of the Cloudinary Customer Advisory Board

Focus on customers has always been Cloudinary’s mantra. Because we owe them our success, we are constantly reaching out to our customers, not just for feedback on our offerings, but also for their vision, wish list, and buy-in of what Cloudinary can do to meet their needs and make them succeed. About six months ago, it occurred to us that it would be beneficial if we could meet regularly with those who are behind innovation at our key customers—executives, product gurus, developers, content managers—to swap strategies, product roadmaps, best practices, and such. In particular, we’d like to solicit actionable feedback as a foundation for our plans of product enhancements.

Read more
Media Management With the Cloudinary-Netlify CMS Integration

Static sites and the JAMstack are quickly becoming a standard for developing safe and performant websites with an optimal workflow for developers. Netlify CMS (not to be confused with the company that created it, Netlify) is an open source content management solution that works especially with static site generators such as Gatsby, Hugo, etc... enabling content storage in your Git repository along with your code for easier versioning, multichannel publishing, and direct content updates in Git.

Read more
Vitaly Friedman's Insights on Media Conferences

Vitaly Friedman is a die-hard devotee of beautiful content. Born in Minsk, Belarus, he studied Computer Science and Mathematics in Germany, unearthing in himself a passion for typography, writing, and design in the interim. After a six-year stint as a freelance designer and developer, he co-founded Smashing Magazine, a leading online publication on web design and development. You can follow SmashingMag on Twitter @SmashingMag.

Read more