Cloudinary Object-Aware Cropping

Cloudinary is a cloud-based service that provides solutions for image and video management. These include server or client-side upload, on-the-fly image and video manipulations, fast CDN delivery, and a variety of asset management options.

The Cloudinary Object-Aware Cropping add-on helps to ensure that your image crops keep the specific objects that matter to you, even when you significantly modify the aspect ratio. This variant of auto-gravity (g_auto) cropping enables you to indicate specific objects or object categories that should be given priority when parts of a photo are cropped out. Cloudinary accomplishes this by applying advanced AI-based object detection algorithms on-the-fly during the crop process.

On this page:

Applying object-aware cropping

After registering for the Cloudinary Object-Aware Cropping add-on, you apply it by specifying an object or an object category as the focal_gravity attribute for the auto gravity parameter (g_auto in URLs) together with a cropping option.

For example, consider the original image of a kitchen below:

Original

You can deliver a square thumbnail crop that centers in on the detected coordinates of the sink, microwave, or refrigerator. To do this, specify the relevant object option for the g_auto gravity definition in conjunction with the thumb cropping option:

URLRubyPHPPythonNode.jsJavaJSjQueryReactVue.jsAngular.NetAndroidiOSAll
URL:
https://res.cloudinary.com/demo/image/upload/w_600,ar_1,c_thumb,g_auto:sink/docs/kitchen1.jpg
Ruby:
cl_image_tag("docs/kitchen1.jpg", :width=>600, :aspect_ratio=>"1", :gravity=>"auto:sink", :crop=>"thumb")
PHP:
cl_image_tag("docs/kitchen1.jpg", array("width"=>600, "aspect_ratio"=>"1", "gravity"=>"auto:sink", "crop"=>"thumb"))
Python:
CloudinaryImage("docs/kitchen1.jpg").image(width=600, aspect_ratio="1", gravity="auto:sink", crop="thumb")
Node.js:
cloudinary.image("docs/kitchen1.jpg", {width: 600, aspect_ratio: "1", gravity: "auto:sink", crop: "thumb"})
Java:
cloudinary.url().transformation(new Transformation().width(600).aspectRatio("1").gravity("auto:sink").crop("thumb")).imageTag("docs/kitchen1.jpg");
JS:
cloudinary.imageTag('docs/kitchen1.jpg', {width: 600, aspectRatio: "1", gravity: "auto:sink", crop: "thumb"}).toHtml();
jQuery:
$.cloudinary.image("docs/kitchen1.jpg", {width: 600, aspect_ratio: "1", gravity: "auto:sink", crop: "thumb"})
React:
<Image publicId="docs/kitchen1.jpg" >
  <Transformation width="600" aspectRatio="1" gravity="auto:sink" crop="thumb" />
</Image>
Vue.js:
<cld-image publicId="docs/kitchen1.jpg" >
  <cld-transformation width="600" aspectRatio="1" gravity="auto:sink" crop="thumb" />
</cld-image>
Angular:
<cl-image public-id="docs/kitchen1.jpg" >
  <cl-transformation width="600" aspect-ratio="1" gravity="auto:sink" crop="thumb">
  </cl-transformation>
</cl-image>
.Net:
cloudinary.Api.UrlImgUp.Transform(new Transformation().Width(600).AspectRatio("1").Gravity("auto:sink").Crop("thumb")).BuildImageTag("docs/kitchen1.jpg")
Android:
MediaManager.get().url().transformation(new Transformation().width(600).aspectRatio("1").gravity("auto:sink").crop("thumb")).generate("docs/kitchen1.jpg");
iOS:
imageView.cldSetImage(cloudinary.createUrl().setTransformation(CLDTransformation().setWidth(600).setAspectRatio("1").setGravity("auto:sink").setCrop("thumb")).generate("docs/kitchen1.jpg")!, cloudinary: cloudinary)

g_auto:sink g_auto:sink g_auto:microwave g_auto:microwave g_auto:refrigerator g_auto:refrigerator

Supported categories and objects

When using the Object-Aware Cropping add-on, you can specify either individual objects or more general object categories.

  • When you specify a category, then the auto-gravity algorithm gives priority any objects that are detected from that category.
  • If you specify the generic object category (g_auto:object), then any detected objects from any category get priority.
  • The regular auto-gravity (subject or classic) behavior also impacts the cropping decision. But if requested objects are detected, they get significantly higher priority than the subjects or salient areas that the regular auto-gravity algorithm selects.

The add-on supports the following categories and objects:

Category Objects
person person
vehicle bicycle, car, motorbike, airplane, bus, train, truck, boat
outdoor traffic_light, stop_sign, parking_meter, fire_hydrant, bench
animal bird, cat, dog, horse, sheep, cow, elephant, bear, zebra, giraffe
accessory backpack, umbrella, handbag, tie, suitcase
sports frisbee, skis, snowboard, sports_ball, kite, baseball_bat, baseball_glove, skateboard, surfboard, tennis_racket
kitchen bottle, wine_glass, cup, fork, knife, spoon, bowl
food banana, apple, sandwich, orange, broccoli, carrot, hotdog, pizza, donut, cake
furniture chair, sofa, potted_plant, bed, dining_table, toilet
electronic tv, laptop, mouse, remote, keyboard, cell_phone
appliance microwave, oven, toaster, sink, refrigerator
indoor book, clock, vase, scissors, teddy_bear, hair_dryer, toothbrush
object any object in any subcategory

Combining focal gravity options

When using auto gravity to determine the area to keep in a crop, you can specify multiple focal_gravity options.

This means that in a single auto-gravity parameter, you can specify:

  • Whether to use the classic or subject auto-gravity algorithm, which in some cases may still have some impact on the exact coordinates of the crop, even if other specified objects or focal gravity options are detected.
  • One or multiple objects (from the same or different categories)
  • Built-in focal gravity options such as face/faces or custom_no_override
  • Other add-on based focal gravity options, such as the adv_face, adv_eyes options from the Advanced Facial Attributes Detection add-on.

For example, your auto-gravity URL parameter might be: g_auto:subject:cat:sofa:faces:adv_eyes

This would instruct the cropping mechanism to use the AI-based subject algorithm, while giving top priority to any cats, sofas, faces, or eyes detected in the photo.

For a complete list of all focal_gravity options, see the auto section of the gravity parameter in the Image Transformation Reference.

Important

  • The focal gravity options can be specified in any order. The order does not impact the result.
  • When multiple items are detected that match the requested focal options, larger, more central, and more in-focus (less blurry) objects will get higher priority.
    In special cases, it's possible further fine-tune this default prioritization. For details, contact support.
  • If a particular image has custom coordinates defined, those coordinates always override all other focal gravity options, unless you use the custom_no_override option in conjunction with the other options.

Choosing the cropping mode

When you specify an object in your auto-gravity parameter, the Object-Aware Cropping AI algorithm detects the coordinates of the object and those coordinates are used by the cropping mode.

  • When using object-based gravity in conjunction with thumb cropping (c_thumb), the image is cropped as closely as possible to the detected coordinates of the object given the requested aspect ratio, and then scales to the requested pixel size.

  • When using object-based gravity with crop mode (c_crop), the detected coordinates are prioritized as the area to keep when determining how much to cut from each edge of the photo in order to achieve the requested pixel size. If the requested pixel size is larger than the coordinates of the detected object, other elements of the image that receive priority from g_auto may impact what else is included in the photo and where in your resulting image the detected object may be located, meaning that the detected object will not necessarily be the center of the photo.

  • When using object-based gravity with any of the fill-based modes (c_fill, c_lfill, c_fill_pad), then the coordinates of the detected object should be retained if any cropping is required after scaling, but other elements of the image that receive priority from g_auto may impact what else is included in the photo and where in your resulting image the detected object may be located, meaning that the detected object will not necessarily be the center of the photo.

The following examples shows how different your cropping results may be for the same requested object in the gravity, but with different cropping modes. In this case, we took the original photo below and applies g_auto:wine_glass with fill, crop, and thumb cropping modes.

Original Original

 

c_fill c_fill c_crop c_crop c_thumb c_thumb

Using object-aware cropping for responsive delivery

You can take advantage of object-based auto-gravity with various cropping modes to assist in responsive art direction. This means that when you deliver different sized images to different devices, you don't just scale the same image, but rather crop images differently for different sizes, so that the important objects are always highly visible.

For example, you may deliver a full-size image to large HD screens, use g_auto:[your_important_object] with fill cropping for medium sized screens, and use g_auto:[your_important_object] with thumb cropping for very small screens.

For more details on delivering responsive images, see the Responsive images guide.

Signed URLs

Cloudinary's dynamic image manipulation URLs are powerful tools. However, due to the potential costs of your customers experimenting with dynamic URLs that apply the object-aware cropping algorithm, image manipulation add-on URLs are required (by default) to be signed using Cloudinary's authenticated API. Alternatively, you can eagerly generate the requested derived images using Cloudinary's authenticated API.

To create a signed delivery URL, set the sign_url parameter to true when building a URL or creating an image tag.

The following code example applies object-aware cropping to the skater image, including a signed Cloudinary URL:

RubyPHPPythonNode.jsJavaJSjQueryReactVue.jsAngular.NetAndroidiOSAll
Ruby:
cl_image_tag("Skater.jpg", :width=>400, :aspect_ratio=>"1", :gravity=>"auto:skateboard", :crop=>"thumb", :sign_url=>true)
PHP:
cl_image_tag("Skater.jpg", array("width"=>400, "aspect_ratio"=>"1", "gravity"=>"auto:skateboard", "crop"=>"thumb", "sign_url"=>true))
Python:
CloudinaryImage("Skater.jpg").image(width=400, aspect_ratio="1", gravity="auto:skateboard", crop="thumb", sign_url=True)
Node.js:
cloudinary.image("Skater.jpg", {width: 400, aspect_ratio: "1", gravity: "auto:skateboard", crop: "thumb", sign_url: true})
Java:
cloudinary.url().transformation(new Transformation().width(400).aspectRatio("1").gravity("auto:skateboard").crop("thumb")).signed(true).imageTag("Skater.jpg");
JS:
cloudinary.imageTag('Skater.jpg', {width: 400, aspectRatio: "1", gravity: "auto:skateboard", crop: "thumb", signUrl: true}).toHtml();
jQuery:
$.cloudinary.image("Skater.jpg", {width: 400, aspect_ratio: "1", gravity: "auto:skateboard", crop: "thumb"})
React:
<Image publicId="Skater.jpg" signUrl="true">
  <Transformation width="400" aspectRatio="1" gravity="auto:skateboard" crop="thumb" />
</Image>
Vue.js:
<cld-image publicId="Skater.jpg" signUrl="true">
  <cld-transformation width="400" aspectRatio="1" gravity="auto:skateboard" crop="thumb" />
</cld-image>
Angular:
<cl-image public-id="Skater.jpg" sign-url="true">
  <cl-transformation width="400" aspect-ratio="1" gravity="auto:skateboard" crop="thumb">
  </cl-transformation>
</cl-image>
.Net:
cloudinary.Api.UrlImgUp.Transform(new Transformation().Width(400).AspectRatio("1").Gravity("auto:skateboard").Crop("thumb")).Signed(true).BuildImageTag("Skater.jpg")
Android:
MediaManager.get().url().transformation(new Transformation().width(400).aspectRatio("1").gravity("auto:skateboard").crop("thumb")).signed(true).generate("Skater.jpg");
iOS:
imageView.cldSetImage(cloudinary.createUrl().setTransformation(CLDTransformation().setWidth(400).setAspectRatio("1").setGravity("auto:skateboard").setCrop("thumb")).generate("Skater.jpg", signUrl: true)!, cloudinary: cloudinary)

The generated Cloudinary URL shown below includes a signature component (/s--acvfjq2y--/). Only URLs with a valid signature that matches the requested image manipulation will be approved for on-the-fly image manipulation and delivery.

https://res.cloudinary.com/my_cloud/image/upload/s--acvfjq2y--/w_400,ar_1,c_thumb,g_auto:skateboard/Skater.jpg

For more details on signed URLs, see Signed delivery URLs.

Note
You can optionally remove the signed URL default requirement for a particular add-on by selecting that add-on in the Allow unsigned add-on transformations section of the Security account settings in the Cloudinary console.