Cloudinary Blog

How to Automatically Remove Photo Backgrounds in Seconds With AI

Remove Photo Backgrounds Automatically with AI

These days, clean and sleek e-Commerce web sites are the way to go. And that almost always includes high quality product photos with a flat white, transparent, or solid background. This is not just a passing fad. There are many practical and well-validated reasons for this choice. The graphic below summarizes a few of these:

Infographic: Why remove photo backgrounds?

But if you are a website or app developer who needs to deliver hundreds or thousands of images, or if your site allows its users to upload product images to your site, getting someone to manually edit photos in a graphic editing tool just won't cut it.

Today, Cloudinary is proud to introduce it's AI Background Removal Add-on, which combines a variety of deep-learning and artificial intelligence algorithms to recognize the primary foreground subject of photos and then accurately remove photo backgrounds in a matter of seconds.

Moments after you upload your photos...

These:

Become These!

Dog on sand - original

Dog on sand - no background

Baseball player - original

baseball player - no background

But of-course you don't have to stop there. Once the original background is gone, add another parameter or two and you've got your own automatic photo background changer tool. How cool is that?

Transparent backgrounds are just a parameter away

Cloudinary is a heavy-duty image and video platform that leverages automation and deep learning algorithms to simplify the entire rich-media workflow for websites and apps -- from the bulk upload of raw photo shots to on-the-fly manipulations for creating multiple variants of media files, automatic optimizations, and responsive delivery across various devices.

The AI Background Removal Add-on can be applied to any image either while uploading it to your Cloudinary account or afterwards, simply by adding the background_removal parameter set to cloudinary_ai in your upload or update method.

To instruct Cloudinary to remove the background from the baseball image shown above while uploading, your upload command (using your chosen Cloudinary SDK) would look something like this:

Ruby:
Cloudinary::Uploader.upload("baseball.jpg",
  :public_id => "baseball_no_bg",
  :background_removal => 'cloudinary_ai',
  :notification_url => "https://mysite.example.com/hooks")
PHP:
\Cloudinary\Uploader::upload("baseball.jpg", 
  array(
    "public_id" => "baseball_no_bg",
    "background_removal" => "cloudinary_ai",
    "notification_url" => "https://mysite.example.com/hooks"));
Python:
cloudinary.uploader.upload("baseball.jpg",
  public_id = "baseball_no_bg",
  background_removal = "cloudinary_ai",
  notification_url = "https://mysite.example.com/hooks")
Node.js:
cloudinary.v2.uploader.upload("baseball.jpg", 
  { public_id: "baseball_no_bg",
    background_removal: "cloudinary_ai",
    notification_url: "https://mysite.example.com/hooks" }),
  function(error, result){console.log(result);});
Java:
cloudinary.uploader().upload("baseball.jpg", 
  ObjectUtils.asMap(
    "public_id", "baseball_no_bg",
    "background_removal", "cloudinary_ai",
    "notification_url", "https://mysite.example.com/hooks"));
.Net:
var uploadParams = new ImageUploadParams(){
  File = new FileDescription(@"baseball.jpg"),
  PublicId = "baseball_no_bg",
  BackgroundRemoval = "cloudinary_ai",
  NotificationUrl = "https://mysite.example.com/hooks"};
var uploadResult = cloudinary.Upload(uploadParams);

Initially, your original image is uploaded to Cloudinary, and then the add-on removes the background and replaces the original with a (transparent background) PNG. The original image is automatically backed up and can be easily recovered if needed.

Note the notification_url used in the above upload command. Even though the background removal process only takes a few seconds, if you plan to deliver the transparent image on your site or app right after it uploads, you can use the notification webhook to check the status of the background removal and potentially display a placeholder image or an in-progress animation until the process is complete.

We only make it look simple

Activating the add-on to erase all traces of the background is really as easy as shown above. But what's going on behind the scenes is anything but simple.

Automatically removing a photo background requires the engine to:

  • Recognize the salient object(s) in the image
  • Accurately segment those object/s
  • Separate the foreground to an alpha layer

The ability to segment objects from an image is a fundamental and well-studied challenge in computer vision. Recently, neural networks have been able to achieve the state-of-the-art results that have been used in applications as varied as autonomous driving, medical image analysis, industrial planning and more.

But that's just the tip of the iceberg for image background removal:

First, you need to determine which objects in the image should be classified as foreground versus the background, which depends on the context and composition of the scene.

Additionally, the standard object segmentation used in most neural networks produces coarse segmentation maps. But for production-level background removal, the segmentation maps of foreground vs. background pixels must be near-perfect to make seamless background removal possible. For instance, fur and hair require special care to segment them properly.

Furthermore, there is currently no significantly large public dataset for accurate background removal, which makes it difficult to acquire the quantity of media assets necessary to train a neural network.

And even while addressing all these challenges, we knew we had to find ways to optimize our algorithm so it could deliver the fast results Cloudinary customers expect on images of any size, and at the huge scales that our customer base requires.

Delivering Optimal Results

Once the add-on removes the background, you can then deliver the resulting transparent PNG as-is, or you may want to add a solid or other neutral background and deliver using Cloudinary's auto‑format (f_auto) and auto-quality (q_auto) optimizations to achieve smaller files while retaining visual quality.

Additionally, the returned transparent image retains the original image dimensions. This means your resulting image may have a lot of white space and may not be centered within those original dimensions. To address this, you can also take advantage of our auto-gravity cropping, to automatically detect the visible item(s) in the image no matter where they are, and then resize and crop to your desired delivery size and aspect ratio without cropping off any of the foreground image that Cloudinary's deep learning algorithm just worked so hard to find.

For example, using the background removal add-on, this cute stuffed sheep returns without the table and wall background, but it sure does have a lot of white space on the right side. And as a transparent PNG, it's quite a fat sheep, weighing in at a bit over 1 MB.

stuffed sheep - original stuffed sheep - no background

But let's deliver our resulting (transparent background) image as a square JPG with a white background, while using f_auto and q_auto to deliver in the most optimized format for the requesting browser. We'll also add g_auto:subject to make sure that no matter what aspect ratio we choose for our crop, the location for the crop is automatically selected based on the main (in this case, the only) subject in the image. For demonstration purposes here, we'll also add a thin gray border.

Ruby:
cl_image_tag("docs/rmv_bgd/stuffed.jpg", :transformation=>[
  {:quality=>"auto", :gravity=>"auto:subject", :height=>1280, :aspect_ratio=>"1", :background=>"white", :crop=>"thumb"},
  {:border=>"1px_solid_gray"}
  ])
PHP:
cl_image_tag("docs/rmv_bgd/stuffed.jpg", array("transformation"=>array(
  array("quality"=>"auto", "gravity"=>"auto:subject", "height"=>1280, "aspect_ratio"=>"1", "background"=>"white", "crop"=>"thumb"),
  array("border"=>"1px_solid_gray")
  )))
Python:
CloudinaryImage("docs/rmv_bgd/stuffed.jpg").image(transformation=[
  {'quality': "auto", 'gravity': "auto:subject", 'height': 1280, 'aspect_ratio': "1", 'background': "white", 'crop': "thumb"},
  {'border': "1px_solid_gray"}
  ])
Node.js:
cloudinary.image("docs/rmv_bgd/stuffed.jpg", {transformation: [
  {quality: "auto", gravity: "auto:subject", height: 1280, aspect_ratio: "1", background: "white", crop: "thumb"},
  {border: "1px_solid_gray"}
  ]})
Java:
cloudinary.url().transformation(new Transformation()
  .quality("auto").gravity("auto:subject").height(1280).aspectRatio("1").background("white").crop("thumb").chain()
  .border("1px_solid_gray")).imageTag("docs/rmv_bgd/stuffed.jpg");
JS:
cloudinary.imageTag('docs/rmv_bgd/stuffed.jpg', {transformation: [
  {quality: "auto", gravity: "auto:subject", height: 1280, aspectRatio: "1", background: "white", crop: "thumb"},
  {border: "1px_solid_gray"}
  ]}).toHtml();
jQuery:
$.cloudinary.image("docs/rmv_bgd/stuffed.jpg", {transformation: [
  {quality: "auto", gravity: "auto:subject", height: 1280, aspect_ratio: "1", background: "white", crop: "thumb"},
  {border: "1px_solid_gray"}
  ]})
React:
<Image publicId="docs/rmv_bgd/stuffed.jpg" >
  <Transformation quality="auto" gravity="auto:subject" height="1280" aspectRatio="1" background="white" crop="thumb" />
  <Transformation border="1px_solid_gray" />
</Image>
Angular:
<cl-image public-id="docs/rmv_bgd/stuffed.jpg" >
  <cl-transformation quality="auto" gravity="auto:subject" height="1280" aspect-ratio="1" background="white" crop="thumb">
  </cl-transformation>
  <cl-transformation border="1px_solid_gray">
  </cl-transformation>
</cl-image>
.Net:
cloudinary.Api.UrlImgUp.Transform(new Transformation()
  .Quality("auto").Gravity("auto:subject").Height(1280).AspectRatio("1").Background("white").Crop("thumb").Chain()
  .Border("1px_solid_gray")).BuildImageTag("docs/rmv_bgd/stuffed.jpg")
Android:
MediaManager.get().url().transformation(new Transformation()
  .quality("auto").gravity("auto:subject").height(1280).aspectRatio("1").background("white").crop("thumb").chain()
  .border("1px_solid_gray")).generate("docs/rmv_bgd/stuffed.jpg");
iOS:
imageView.cldSetImage(cloudinary.createUrl().setTransformation(CLDTransformation()
  .setQuality("auto").setGravity("auto:subject").setHeight(1280).setAspectRatio("1").setBackground("white").setCrop("thumb").chain()
  .setBorder("1px_solid_gray")).generate("docs/rmv_bgd/stuffed.jpg")!, cloudinary: cloudinary)
Sheep JPG with white background

Now we get our sheep with a solid-white background, and even if you were to retain the original 1280 px height, the delivered image would still only weigh 92kb.

Another great way you can deliver your image with no excess whitespace is to apply Cloudinary's trim transformation effect, which trims off all excess background:

Ruby:
cl_image_tag("docs/rmv_bgd/stuffed.jpg", :transformation=>[
  {:quality=>"auto", :effect=>"trim", :background=>"white"},
  {:border=>"1px_solid_gray"}
  ])
PHP:
cl_image_tag("docs/rmv_bgd/stuffed.jpg", array("transformation"=>array(
  array("quality"=>"auto", "effect"=>"trim", "background"=>"white"),
  array("border"=>"1px_solid_gray")
  )))
Python:
CloudinaryImage("docs/rmv_bgd/stuffed.jpg").image(transformation=[
  {'quality': "auto", 'effect': "trim", 'background': "white"},
  {'border': "1px_solid_gray"}
  ])
Node.js:
cloudinary.image("docs/rmv_bgd/stuffed.jpg", {transformation: [
  {quality: "auto", effect: "trim", background: "white"},
  {border: "1px_solid_gray"}
  ]})
Java:
cloudinary.url().transformation(new Transformation()
  .quality("auto").effect("trim").background("white").chain()
  .border("1px_solid_gray")).imageTag("docs/rmv_bgd/stuffed.jpg");
JS:
cloudinary.imageTag('docs/rmv_bgd/stuffed.jpg', {transformation: [
  {quality: "auto", effect: "trim", background: "white"},
  {border: "1px_solid_gray"}
  ]}).toHtml();
jQuery:
$.cloudinary.image("docs/rmv_bgd/stuffed.jpg", {transformation: [
  {quality: "auto", effect: "trim", background: "white"},
  {border: "1px_solid_gray"}
  ]})
React:
<Image publicId="docs/rmv_bgd/stuffed.jpg" >
  <Transformation quality="auto" effect="trim" background="white" />
  <Transformation border="1px_solid_gray" />
</Image>
Angular:
<cl-image public-id="docs/rmv_bgd/stuffed.jpg" >
  <cl-transformation quality="auto" effect="trim" background="white">
  </cl-transformation>
  <cl-transformation border="1px_solid_gray">
  </cl-transformation>
</cl-image>
.Net:
cloudinary.Api.UrlImgUp.Transform(new Transformation()
  .Quality("auto").Effect("trim").Background("white").Chain()
  .Border("1px_solid_gray")).BuildImageTag("docs/rmv_bgd/stuffed.jpg")
Android:
MediaManager.get().url().transformation(new Transformation()
  .quality("auto").effect("trim").background("white").chain()
  .border("1px_solid_gray")).generate("docs/rmv_bgd/stuffed.jpg");
iOS:
imageView.cldSetImage(cloudinary.createUrl().setTransformation(CLDTransformation()
  .setQuality("auto").setEffect("trim").setBackground("white").chain()
  .setBorder("1px_solid_gray")).generate("docs/rmv_bgd/stuffed.jpg")!, cloudinary: cloudinary)
Sheep JPG with white background

Everything isn't always black and white

As you can see, optimized image delivery will often involve replacing the transparent background and delivering in a format other than the PNG that our background removal add-on initially returns. But there are good reasons why we don't just return a solid white JPG to begin with.

Going neutral

For starters, delivering with a light, neutral background color can provide the same advantages as white, but it can break the monotony of an all-white page, and even communicate a certain style or personality without taking the focus away from the product. In our research, we've seen a variety of retailers using that strategy. For example, here's the approach Walmart.com has taken on their apparel landing page:

Walmart example

To automatically change the background color with Cloudinary, just change the value of the background (b_) parameter in your URL to the desired color. For example, check out our sheep with a very light blue background.

True colors

Conversely, we've seen some retailers going for brightly colored backgrounds when presenting white or black products, while sticking with neutral backgrounds with their other products, like this example from Zara.com:

Zara example

Without a shadow of a doubt

Starting with a fully transparent background also makes it easy to add shadows. Of-course you may have already had some sort of shadow in the original photograph, but our Background Removal add-on removes that shadow, enabling you to add a consistent shadow location and size for all your products.

For example:

Original remote control image Original remote control image  No background No background No background + added shadow No background + added shadow

Here's the transformation we used above to add a shadow to the background-less remote control:

Ruby:
cl_image_tag("docs/rmv_bgd/remote.png", :transformation=>[
  {:width=>200, :crop=>"scale"},
  {:effect=>"shadow:50", :x=>6, :y=>6}
  ])
PHP:
cl_image_tag("docs/rmv_bgd/remote.png", array("transformation"=>array(
  array("width"=>200, "crop"=>"scale"),
  array("effect"=>"shadow:50", "x"=>6, "y"=>6)
  )))
Python:
CloudinaryImage("docs/rmv_bgd/remote.png").image(transformation=[
  {'width': 200, 'crop': "scale"},
  {'effect': "shadow:50", 'x': 6, 'y': 6}
  ])
Node.js:
cloudinary.image("docs/rmv_bgd/remote.png", {transformation: [
  {width: 200, crop: "scale"},
  {effect: "shadow:50", x: 6, y: 6}
  ]})
Java:
cloudinary.url().transformation(new Transformation()
  .width(200).crop("scale").chain()
  .effect("shadow:50").x(6).y(6)).imageTag("docs/rmv_bgd/remote.png");
JS:
cloudinary.imageTag('docs/rmv_bgd/remote.png', {transformation: [
  {width: 200, crop: "scale"},
  {effect: "shadow:50", x: 6, y: 6}
  ]}).toHtml();
jQuery:
$.cloudinary.image("docs/rmv_bgd/remote.png", {transformation: [
  {width: 200, crop: "scale"},
  {effect: "shadow:50", x: 6, y: 6}
  ]})
React:
<Image publicId="docs/rmv_bgd/remote.png" >
  <Transformation width="200" crop="scale" />
  <Transformation effect="shadow:50" x="6" y="6" />
</Image>
Angular:
<cl-image public-id="docs/rmv_bgd/remote.png" >
  <cl-transformation width="200" crop="scale">
  </cl-transformation>
  <cl-transformation effect="shadow:50" x="6" y="6">
  </cl-transformation>
</cl-image>
.Net:
cloudinary.Api.UrlImgUp.Transform(new Transformation()
  .Width(200).Crop("scale").Chain()
  .Effect("shadow:50").X(6).Y(6)).BuildImageTag("docs/rmv_bgd/remote.png")
Android:
MediaManager.get().url().transformation(new Transformation()
  .width(200).crop("scale").chain()
  .effect("shadow:50").x(6).y(6)).generate("docs/rmv_bgd/remote.png");
iOS:
imageView.cldSetImage(cloudinary.createUrl().setTransformation(CLDTransformation()
  .setWidth(200).setCrop("scale").chain()
  .setEffect("shadow:50").setX(6).setY(6)).generate("docs/rmv_bgd/remote.png")!, cloudinary: cloudinary)

Green-screening without a green screen

If you want to have some fun (or pass that fun on to your users), you can add a fun background scene behind your transparent image for an auto photo background changer/green-screen-like effect, regardless of the original background. Just use the underlay parameter set to the public ID of any image in your Cloudinary account as the new background layer.

For example, with an instantaneous on-the-fly transformation, we could send our baseball player to outer space, get the dog off the couch, or place hiking boots product in a more nature-oriented environment:

Baseball player - original photo Baseball player in outer space

Lazy dog - original photo Lazy dog at the beach

Hiking boots - original photo Hiking boots in nature

Below is the delivery code for the hiking boots example.
(Click the other transformed images on the right above to view their delivery URLs.)

Ruby:
cl_image_tag("docs/rmv_bgd/shoes.png", :transformation=>[
  {:width=>800, :quality=>"auto", :crop=>"scale"},
  {:underlay=>"docs:canyon2", :gravity=>"south", :height=>1000}
  ])
PHP:
cl_image_tag("docs/rmv_bgd/shoes.png", array("transformation"=>array(
  array("width"=>800, "quality"=>"auto", "crop"=>"scale"),
  array("underlay"=>"docs:canyon2", "gravity"=>"south", "height"=>1000)
  )))
Python:
CloudinaryImage("docs/rmv_bgd/shoes.png").image(transformation=[
  {'width': 800, 'quality': "auto", 'crop': "scale"},
  {'underlay': "docs:canyon2", 'gravity': "south", 'height': 1000}
  ])
Node.js:
cloudinary.image("docs/rmv_bgd/shoes.png", {transformation: [
  {width: 800, quality: "auto", crop: "scale"},
  {underlay: "docs:canyon2", gravity: "south", height: 1000}
  ]})
Java:
cloudinary.url().transformation(new Transformation()
  .width(800).quality("auto").crop("scale").chain()
  .underlay(new Layer().publicId("docs:canyon2")).gravity("south").height(1000)).imageTag("docs/rmv_bgd/shoes.png");
JS:
cloudinary.imageTag('docs/rmv_bgd/shoes.png', {transformation: [
  {width: 800, quality: "auto", crop: "scale"},
  {underlay: new cloudinary.Layer().publicId("docs:canyon2"), gravity: "south", height: 1000}
  ]}).toHtml();
jQuery:
$.cloudinary.image("docs/rmv_bgd/shoes.png", {transformation: [
  {width: 800, quality: "auto", crop: "scale"},
  {underlay: new cloudinary.Layer().publicId("docs:canyon2"), gravity: "south", height: 1000}
  ]})
React:
<Image publicId="docs/rmv_bgd/shoes.png" >
  <Transformation width="800" quality="auto" crop="scale" />
  <Transformation underlay="docs:canyon2" gravity="south" height="1000" />
</Image>
Angular:
<cl-image public-id="docs/rmv_bgd/shoes.png" >
  <cl-transformation width="800" quality="auto" crop="scale">
  </cl-transformation>
  <cl-transformation underlay="docs:canyon2" gravity="south" height="1000">
  </cl-transformation>
</cl-image>
.Net:
cloudinary.Api.UrlImgUp.Transform(new Transformation()
  .Width(800).Quality("auto").Crop("scale").Chain()
  .Underlay(new Layer().PublicId("docs:canyon2")).Gravity("south").Height(1000)).BuildImageTag("docs/rmv_bgd/shoes.png")
Android:
MediaManager.get().url().transformation(new Transformation()
  .width(800).quality("auto").crop("scale").chain()
  .underlay(new Layer().publicId("docs:canyon2")).gravity("south").height(1000)).generate("docs/rmv_bgd/shoes.png");
iOS:
imageView.cldSetImage(cloudinary.createUrl().setTransformation(CLDTransformation()
  .setWidth(800).setQuality("auto").setCrop("scale").chain()
  .setUnderlay("docs:canyon2").setGravity("south").setHeight(1000)).generate("docs/rmv_bgd/shoes.png")!, cloudinary: cloudinary)

Putting it all together

Whether you are delivering product photos on your own site or on multiple marketplaces, or you want to offer background removal services for your end users' uploaded photos, you need an automated solution that will quickly and reliably remove photo backgrounds, leaving you with a sharp and clear foreground image.

Once the original background is 'out of the picture', you can freely deliver your product images with white backgrounds, neutral or even bright colors, or you can put them in context of a realistic scene.

In fact, you may want to do all of the above. One way to do this is to put all your resulting background-free images together, along with other views such as 3D images and videos, in a single carousel-controlled gallery, using Cloudinary's easy-to-implement Product Gallery. You can learn more in the Product Gallery blog post, video tutorial, and documentation.

Your turn!

To get started removing backgrounds yourself, just sign up for a free Cloudinary account, register for the free trial tier of our Cloudinary AI Background Removal add-on, and check out the Background Removal add-on documentation.

We'd love to see your background removal photo results in the comments!

Recent Blog Posts

Video Manipulations and Delivery for Angular Video Apps

On social media, videos posted by users constitute a significant amount of the content appeal on those platforms. From upload to manipulation to delivery, a smooth, efficient, and effective pipeline for the posting process is mandatory to ensure consistent user sessions and their steadily increasing volume. However, building such an infrastructure is a complex, labor-intensive, and problem-prone undertaking.

Read more
Green Screen Queen: Dynamic Video Transparency Fit For Royalty

If you were reading your social media or news feeds on or around June 11 this year, no doubt you came across your fair share of posts about Queen Elizabeth and her outfit-color faux pas. For her 90th birthday, she chose a solid neon green suit, and it didn't take long for Photoshop fanatics to suggest alternative designs for the Queen's green-screen threads.

Read more
Content-Aware Automatic Cropping for Video

Delivering videos according to the aspect ratios defined by social media for multiple devices and platforms is a growing challenge. The continuously rising volume of vertical videos and the corresponding increase in video traffic on mobile devices (now up to 57% of online videos watched) have only exacerbated the situation, with no letup in sight.

Read more
Use a custom function in the image delivery pipeline

Cloudinary offers a wide array of image manipulations and effects to apply to images as part of our image-processing pipeline, helping to ensure that your images fit the graphic design of your website or mobile application. Cloudinary is an open platform, and you can use our APIs, Widgets and UI to build the media management flow that matches your needs.

Read more
OpenText™ TeamSite – Cloudinary Plugin

Tired of depending on other teams or software to create assets in multiple sizes for your responsive web site?

Does importing asset files into TeamSite slow down your web content publishing?

Klish Group is pleased to introduce the OpenText TM TeamSite – Cloudinary connector. Customers of the OpenText TM TeamSite web content management platform can now browse and select images in the same way they always have. Authors can just browse and select the image they want to use and Cloudinary will automatically deliver it in the optimal format and quality to the customer requesting it.

Read more