Cloudinary AI Background Removal

Cloudinary is a cloud-based service that provides solutions for image and video management, including server or client-side upload, on-the-fly image and video manipulations, quick CDN delivery, and a variety of asset management options.

The Cloudinary AI Background Removal add-on combines a variety of deep-learning algorithms to recognize the primary foreground object(s) in a photo and accurately remove the background in a matter of seconds. You can optionally specify one of a set of object names to instruct the add-on to remove everything except that object.

Activating the add-on

The default background removal behavior detects the foreground objects(s) of the image and removes the background. You activate this behavior by setting the background_removal parameter to cloudinary_ai when uploading an image (Upload method) or by using the Update method of the Admin API for existing images.

You can also include additional directives for indicating what the AI algorithm should treat as the foreground object to keep. For details see: Specifying the foreground object below.

The background removal process only takes a few seconds, but is still handled asynchronously, so you may want to include a Notification in your method call by specifying the notification_url parameter.

Removing the background while uploading

The code below uploads the photo shown below on the left and activates the Cloudinary AI background removal add-on. The photo that ultimately gets stored in the account is the dog image shown on the right, with a fully transparent background.

Ruby:
Copy to clipboard
Cloudinary::Uploader.upload("dog_couch.jpg",
  :public_id => "dog_couch",
  :background_removal => 'cloudinary_ai',
  :notification_url => "https://mysite.example.com/hooks")
PHP:
Copy to clipboard
\Cloudinary\Uploader::upload("dog_couch.jpg", 
  array(
    "public_id" => "dog_couch",
    "background_removal" => "cloudinary_ai",
    "notification_url" => "https://mysite.example.com/hooks"));
Python:
Copy to clipboard
cloudinary.uploader.upload("dog_couch.jpg",
  public_id = "dog_couch",
  background_removal = "cloudinary_ai",
  notification_url = "https://mysite.example.com/hooks")
Node.js:
Copy to clipboard
cloudinary.v2.uploader.upload("dog_couch.jpg", 
  { public_id: "dog_couch",
    background_removal: "cloudinary_ai",
    notification_url: "https://mysite.example.com/hooks" },
  function(error, result){console.log(result);});
Java:
Copy to clipboard
cloudinary.uploader().upload("dog_couch.jpg", 
  ObjectUtils.asMap(
    "public_id", "dog_couch",
    "background_removal", "cloudinary_ai",
    "notification_url", "https://mysite.example.com/hooks"));
.Net:
Copy to clipboard
var uploadParams = new ImageUploadParams(){
  File = new FileDescription(@"dog_couch.jpg"),
  PublicId = "dog_couch",
  BackgroundRemoval = "cloudinary_ai",
  NotificationUrl = "https://mysite.example.com/hooks"};
var uploadResult = cloudinary.Upload(uploadParams);
iOS:
Copy to clipboard
let params = CLDUploadRequestParams()
  .setPublicId("dog_couch")
  .setBackgroundRemoval("cloudinary_ai")
  .setNotificationUrl("https://mysite.example.com/hooks")
var mySig = MyFunction(params)  // your own function that returns a signature generated on your backend
params.setSignature(CLDSignature(signature: mySig.signature, timestamp: mySig.timestamp))
let request = cloudinary.createUploader().signedUpload(
  url: "dog_couch.jpg", params: params)
Original image to upload Original image to upload Uploaded image with background removal add-on applied Uploaded image with the add-on applied

Tip
You can use upload presets to centrally define a set of upload options including add-on operations to apply, instead of specifying them in each upload call. You can define multiple upload presets, and apply different presets in different upload scenarios. You can create new upload presets in the Upload page of the Management Console settings or using the upload_presets Admin API method. From the Upload page of the console settings, you can also select default upload presets to use for image, video, and raw API uploads (respectively) as well as default presets for image, video, and raw uploads performed via the Media Library UI.

Removing the background from existing images

You can remove the background from existing images, using the Update Admin API method:

Ruby:
Copy to clipboard
Cloudinary::Api.update("woman", 
  :background_removal => "cloudinary_ai",
  :notification_url => "https://mysite.example.com/hooks")
PHP:
Copy to clipboard
$api = new \Cloudinary\Api();
$api->update("woman", 
  array(
  "background_removal" => "cloudinary_ai",
  "notification_url" => "https://mysite.example.com/hooks"));
Python:
Copy to clipboard
cloudinary.api.update("woman",
  background_removal = "cloudinary_ai",
  notification_url = "https://mysite.example.com/hooks")
Node.js:
Copy to clipboard
cloudinary.v2.api.update("woman", 
  { background_removal: "cloudinary_ai",
    notification_url: "https://mysite.example.com/hooks" },
  function(error, result){console.log(result);});
Java:
Copy to clipboard
cloudinary.api().update("woman", 
  ObjectUtils.asMap(
    "background_removal", "cloudinary_ai",
    "notification_url", "https://mysite.example.com/hooks" ));
.Net:
Copy to clipboard
var updateParams = new UpdateParams("woman"){
  BackgroundRemoval = "cloudinary_ai",
  NotificationUrl = "https://mysite.example.com/hooks"};
var updateResult = cloudinary.UpdateResource(updateParams);

Specifying the foreground object

In some cases, the AI algorithm may identify more than one object in the foreground. If you know which type of item you expect in the photos you are applying the add-on to, and that item is among the supported objects listed below, you can request to keep only object(s) of that type and treat everything else as background.

To do this, set the background_removal parameter to cloudinary_ai:[object_to_keep] when uploading or updating an image.

For example, in the original image below, both the cars and the roadsign are identified by the add-on algorithm as foreground objects. However, by specifying background_removal:"cloudinary_ai:car" in the upload or update method, the sign is also treated as part of the background, and only the cars remain:

Original Original Default background removal Default background removal cloudinary_ai:car cloudinary_ai:car

Similarly, if you want to deliver images containing only dogs or only cats, you can ensure you only keep the animal you want:

Original Original
Default background removal Default background removal cloudinary_ai:cat cloudinary_ai:cat cloudinary_ai:dog cloudinary_ai:dog

Supported foreground objects

You can specify any of the following as the foreground object to keep when you remove the background:

airplane bus dining_table sheep
bicycle car dog sofa
bird cat horse train
boat chair person tv
bottle cow potted_plant motorbike

Defining the edges of the foreground object

If you know that your image has a foreground object with fine detail around its edges, for example hair, you can request to keep this detail.

To do this, set the background_removal parameter to cloudinary_ai:fine_edges when uploading or updating an image.

You should not use this option if you know that the object has more clear-cut edges.

Ruby:
Copy to clipboard
Cloudinary::Uploader.upload("girl_hair.jpg",
  :public_id => "girl_hair_fine",
  :background_removal => 'cloudinary_ai:fine_edges',
  :notification_url => "https://mysite.example.com/hooks")
PHP:
Copy to clipboard
\Cloudinary\Uploader::upload("girl_hair.jpg", 
  array(
    "public_id" => "girl_hair_fine",
    "background_removal" => "cloudinary_ai:fine_edges",
    "notification_url" => "https://mysite.example.com/hooks"));
Python:
Copy to clipboard
cloudinary.uploader.upload("girl_hair.jpg",
  public_id = "girl_hair_fine",
  background_removal = "cloudinary_ai:fine_edges",
  notification_url = "https://mysite.example.com/hooks")
Node.js:
Copy to clipboard
cloudinary.v2.uploader.upload("girl_hair.jpg", 
  { public_id: "girl_hair_fine",
    background_removal: "cloudinary_ai:fine_edges",
    notification_url: "https://mysite.example.com/hooks" },
  function(error, result){console.log(result);});
Java:
Copy to clipboard
cloudinary.uploader().upload("girl_hair.jpg", 
  ObjectUtils.asMap(
    "public_id", "girl_hair_fine",
    "background_removal", "cloudinary_ai:fine_edges",
    "notification_url", "https://mysite.example.com/hooks"));
.Net:
Copy to clipboard
var uploadParams = new ImageUploadParams(){
  File = new FileDescription(@"girl_hair.jpg"),
  PublicId = "girl_hair_fine",
  BackgroundRemoval = "cloudinary_ai:fine_edges",
  NotificationUrl = "https://mysite.example.com/hooks"};
var uploadResult = cloudinary.Upload(uploadParams);
iOS:
Copy to clipboard
let params = CLDUploadRequestParams()
  .setPublicId("girl_hair_fine")
  .setBackgroundRemoval("cloudinary_ai:fine_edges")
  .setNotificationUrl("https://mysite.example.com/hooks")
var mySig = MyFunction(params)  // your own function that returns a signature generated on your backend
params.setSignature(CLDSignature(signature: mySig.signature, timestamp: mySig.timestamp))
let request = cloudinary.createUploader().signedUpload(
  url: "girl_hair.jpg", params: params)
Original Original Default background removal Default background removal cloudinary_ai:fine_edges cloudinary_ai:fine_edges

You can see the difference more clearly when you zoom in:

Original Original Default background removal Default background removal cloudinary_ai:fine_edges cloudinary_ai:fine_edges

Asynchronous handling

Although the background removal algorithm usually takes only a few seconds, it is still handled asynchronously after the original file is uploaded or updated. Therefore, the immediate upload response indicates that the background_removal status is pending. For example, here's the upload response from the above dog-on-couch image.

Copy to clipboard
{
  "public_id": "dog_couch",
  "version": 1551099901,
   ...
   ...
  "url": "http://res.cloudinary.com/demo/image/upload/v1551101478/dog_couch.jpg",
  "secure_url": "https://res.cloudinary.com/demo/image/upload/v1551101478/dog_couch.jpg",
  "info": {
    "background_removal": {
      "cloudinary_ai": {
        "status": "pending"
      }
    }
  },
  "original_filename": "dog_couch"
}

Immediately after you perform the upload or update method, the image in your account is your original image. When the background removal process is complete, the original image is overwritten with a PNG containing the foreground image and transparent background.

If you requested a notification in your upload or update call by adding the notification_url parameter, the endpoint you specified will receive a JSON POST request when the background removal is complete, including the new url & secure_url with the updated .png file extension and new version number:

Copy to clipboard
{
    "info_kind": "cloudinary_ai",
    "info_status": "complete",
    "public_id": "dog_couch",
    "uploaded_at": "2019-02-25T17:33:45Z",
    "version": 1551104931,
    "url": "http://res.cloudinary.com/demo/image/upload/v1551104931/dog_couch.png",
    "secure_url": "https://res.cloudinary.com/demo/image/upload/v1551104931/dog_couch.png",
    "etag": "6567d798ca4087468dc7d23bcb8a45ec",
    "notification_type": "info"
}

Delivering and transforming your new image

After the background has been removed from your image, you can take advantage of a variety of image transformations to deliver the new image to your users.

Use case #1 - Add a shadow to a product image

If all the product images in your online store have a shadow, then after you've applied the Cloudinary AI Remove the Background add-on to this router photo, you could use the transformation code below to apply a shadow to the delivered image.

Ruby:
Copy to clipboard
cl_image_tag("docs/rmv_bgd/router.png", :transformation=>[
  {:height=>105, :crop=>"scale"},
  {:effect=>"shadow:50", :x=>10, :y=>10}
  ])
PHP:
Copy to clipboard
cl_image_tag("docs/rmv_bgd/router.png", array("transformation"=>array(
  array("height"=>105, "crop"=>"scale"),
  array("effect"=>"shadow:50", "x"=>10, "y"=>10)
  )))
Python:
Copy to clipboard
CloudinaryImage("docs/rmv_bgd/router.png").image(transformation=[
  {'height': 105, 'crop': "scale"},
  {'effect': "shadow:50", 'x': 10, 'y': 10}
  ])
Node.js:
Copy to clipboard
cloudinary.image("docs/rmv_bgd/router.png", {transformation: [
  {height: 105, crop: "scale"},
  {effect: "shadow:50", x: 10, y: 10}
  ]})
Java:
Copy to clipboard
cloudinary.url().transformation(new Transformation()
  .height(105).crop("scale").chain()
  .effect("shadow:50").x(10).y(10)).imageTag("docs/rmv_bgd/router.png");
JS:
Copy to clipboard
cloudinary.imageTag('docs/rmv_bgd/router.png', {transformation: [
  {height: 105, crop: "scale"},
  {effect: "shadow:50", x: 10, y: 10}
  ]}).toHtml();
jQuery:
Copy to clipboard
$.cloudinary.image("docs/rmv_bgd/router.png", {transformation: [
  {height: 105, crop: "scale"},
  {effect: "shadow:50", x: 10, y: 10}
  ]})
React:
Copy to clipboard
<Image publicId="docs/rmv_bgd/router.png" >
  <Transformation height="105" crop="scale" />
  <Transformation effect="shadow:50" x="10" y="10" />
</Image>
Vue.js:
Copy to clipboard
<cld-image publicId="docs/rmv_bgd/router.png" >
  <cld-transformation height="105" crop="scale" />
  <cld-transformation effect="shadow:50" x="10" y="10" />
</cld-image>
Angular:
Copy to clipboard
<cl-image public-id="docs/rmv_bgd/router.png" >
  <cl-transformation height="105" crop="scale">
  </cl-transformation>
  <cl-transformation effect="shadow:50" x="10" y="10">
  </cl-transformation>
</cl-image>
.Net:
Copy to clipboard
cloudinary.Api.UrlImgUp.Transform(new Transformation()
  .Height(105).Crop("scale").Chain()
  .Effect("shadow:50").X(10).Y(10)).BuildImageTag("docs/rmv_bgd/router.png")
Android:
Copy to clipboard
MediaManager.get().url().transformation(new Transformation()
  .height(105).crop("scale").chain()
  .effect("shadow:50").x(10).y(10)).generate("docs/rmv_bgd/router.png");
iOS:
Copy to clipboard
imageView.cldSetImage(cloudinary.createUrl().setTransformation(CLDTransformation()
  .setHeight(105).setCrop("scale").chain()
  .setEffect("shadow:50").setX(10).setY(10)).generate("docs/rmv_bgd/router.png")!, cloudinary: cloudinary)

Original router image Original router image Router without background No background No background + shadow No background + shadow

Use case #2 - Apply green screen style backdrops

You can take advantage of the Cloudinary AI Remove the Background add-on to provide an app with fun effects. For example, you could allow users to upload images of themselves and then select new background locations by adding the desired scenery as an underlay layer behind your transparent background.

For example, after running the add-on on the original image to remove the background, you could use the following delivery code to relocate this woman from her office to the beach:

Ruby:
Copy to clipboard
cl_image_tag("docs/rmv_bgd/woman.png", :transformation=>[
  {:height=>375, :crop=>"scale"},
  {:underlay=>"docs:bg_beach", :gravity=>"south", :width=>800}
  ])
PHP:
Copy to clipboard
cl_image_tag("docs/rmv_bgd/woman.png", array("transformation"=>array(
  array("height"=>375, "crop"=>"scale"),
  array("underlay"=>"docs:bg_beach", "gravity"=>"south", "width"=>800)
  )))
Python:
Copy to clipboard
CloudinaryImage("docs/rmv_bgd/woman.png").image(transformation=[
  {'height': 375, 'crop': "scale"},
  {'underlay': "docs:bg_beach", 'gravity': "south", 'width': 800}
  ])
Node.js:
Copy to clipboard
cloudinary.image("docs/rmv_bgd/woman.png", {transformation: [
  {height: 375, crop: "scale"},
  {underlay: "docs:bg_beach", gravity: "south", width: 800}
  ]})
Java:
Copy to clipboard
cloudinary.url().transformation(new Transformation()
  .height(375).crop("scale").chain()
  .underlay(new Layer().publicId("docs:bg_beach")).gravity("south").width(800)).imageTag("docs/rmv_bgd/woman.png");
JS:
Copy to clipboard
cloudinary.imageTag('docs/rmv_bgd/woman.png', {transformation: [
  {height: 375, crop: "scale"},
  {underlay: new cloudinary.Layer().publicId("docs:bg_beach"), gravity: "south", width: 800}
  ]}).toHtml();
jQuery:
Copy to clipboard
$.cloudinary.image("docs/rmv_bgd/woman.png", {transformation: [
  {height: 375, crop: "scale"},
  {underlay: new cloudinary.Layer().publicId("docs:bg_beach"), gravity: "south", width: 800}
  ]})
React:
Copy to clipboard
<Image publicId="docs/rmv_bgd/woman.png" >
  <Transformation height="375" crop="scale" />
  <Transformation underlay="docs:bg_beach" gravity="south" width="800" />
</Image>
Vue.js:
Copy to clipboard
<cld-image publicId="docs/rmv_bgd/woman.png" >
  <cld-transformation height="375" crop="scale" />
  <cld-transformation underlay="docs:bg_beach" gravity="south" width="800" />
</cld-image>
Angular:
Copy to clipboard
<cl-image public-id="docs/rmv_bgd/woman.png" >
  <cl-transformation height="375" crop="scale">
  </cl-transformation>
  <cl-transformation underlay="docs:bg_beach" gravity="south" width="800">
  </cl-transformation>
</cl-image>
.Net:
Copy to clipboard
cloudinary.Api.UrlImgUp.Transform(new Transformation()
  .Height(375).Crop("scale").Chain()
  .Underlay(new Layer().PublicId("docs:bg_beach")).Gravity("south").Width(800)).BuildImageTag("docs/rmv_bgd/woman.png")
Android:
Copy to clipboard
MediaManager.get().url().transformation(new Transformation()
  .height(375).crop("scale").chain()
  .underlay(new Layer().publicId("docs:bg_beach")).gravity("south").width(800)).generate("docs/rmv_bgd/woman.png");
iOS:
Copy to clipboard
imageView.cldSetImage(cloudinary.createUrl().setTransformation(CLDTransformation()
  .setHeight(375).setCrop("scale").chain()
  .setUnderlay("docs:bg_beach").setGravity("south").setWidth(800)).generate("docs/rmv_bgd/woman.png")!, cloudinary: cloudinary)

Original personal photo Original personal photo No background No background

 

No background + rainbow scene No background + rainbow scene No background + beach scene No background + beach scene

Guidelines and Best Practices

This add-on is based on a combination of neural networks that were trained on a large dataset to create precise segmentation maps of salient objects and refine those maps to accurately separate the edges of the foreground from the background. These neural networks were also optimized to enable returning the result within seconds regardless of the image size.1

In the great majority of cases, these deep-learning algorithms return high quality results. The following best practices can further increase the liklihood of good results:

  • Make sure there is enough background around the foreground object(s) on all sides.
  • Try to avoid large shadows. This is especially relevant if the background is a single color. For example, if the main subject is photographed against a plain light colored background, the object's shadow might be classified as part of the foreground.
  • In some cases, hair or other very small or blurry parts on the edges of a foreground object might be blended with the background. For example, try to avoid hairs blowing in the wind.
  • In some cases, strong changes in contrast, such as a bright lamp or a bright sun included in the photo, can impact the results.
  • Light-colored reflections on a shiny object (e.g. shine from the camera flash) may be treated as part of the background and removed, especially if the color of the reflection is similar to the main background color.

However, as with any Deep Learning-based algorithm, even if you follow these guidelines, the results may not always be perfect. Additionally, keep in mind that the neural networks are continually training on new data and thus results may be different over time.

If you are not satisfied with the results of a background removal operation, you can use one (or both) of the following options:

  • Restore from backup: When the newly generated (removed background) image overwrites the old one, the original image is backed up and is accessible from the View backed up versions button, available when you open the Edit Transformation page for that image in the Media Library. This backup is performed for all images where the Cloudinary AI Remove the Background add-on is applied, even if you haven't enabled backups globally for your account.

Originals are stored as backups when you activate the AI Remove the Background add-on

  • Pixelz Remove the background add-on: The Pixelz Remove the Background add-on automatically uploads your original image to the Pixelz team of human experts who manually remove the background of your image within 24 hours. When complete, the new image replaces your original one in your account similar to the behavior of the Cloudinary AI Remove the Background add-on. You can monitor the status using a notification_url.

1The add-on does not have any specific file or resolution size limitations. But keep in mind that the returned file is always a PNG with transparent background. Even if the originally uploaded image is within your file size account limits, large resolution images may result in very large file sizes for the returned PNG, and may exceed the max file size limits for your account. If the returned file exceeds your account limits, then it will not be stored in your account and the original will remain.

✔️ Feedback sent!