{"id":22099,"date":"2020-07-06T15:25:49","date_gmt":"2020-07-06T15:25:49","guid":{"rendered":"http:\/\/the_object_oriented_approach_to_automatic_image_cropping"},"modified":"2024-06-04T16:09:49","modified_gmt":"2024-06-04T23:09:49","slug":"the_object_oriented_approach_to_automatic_image_cropping","status":"publish","type":"post","link":"https:\/\/cloudinary.com\/blog\/the_object_oriented_approach_to_automatic_image_cropping","title":{"rendered":"The Object-Oriented Approach to Automatic Image Cropping"},"content":{"rendered":"

If you\u2019ve come across this post and, based on the title, are expecting insights into object-oriented programming languages, then please forgive me for overloading the term object-oriented<\/em>. I hope you don\u2019t object when you discover that the objects in this post are not so much the instantiated class types, but real-life objects found in images. It is helpful though if you\u2019re familiar with the likes of Java, PHP, Javascript or other popular frameworks<\/a>, as you\u2019ll be able to implement these automatic image cropping techniques in your code. Anyway, let\u2019s cut to the chase – or should that be crop to the object<\/em>\u2026?<\/p>\n

Automatic AI-driven object-aware cropping is a very powerful feature that can save you time, money and a lot of hassle when it comes to displaying images on your website or app. For example, a typical e-commerce website may have 10,000 products in its catalog, with four images per product. So, unless AI is used, it\u2019s an expensive proposition to ensure that 40,000 images of different sizes and aspect ratios best portray the objects you are selling.<\/p>\n

Cloudinary\u2019s Object-Aware Cropping add-on<\/a> applies advanced AI-based object detection algorithms on-the-fly during the crop process to keep specific objects in the picture. Like many of Cloudinary\u2019s image transformation<\/a> features, all you need to do is add a few parameters to the image delivery URL. In this case, you specify what you want to keep in your image along with one of Cloudinary\u2019s many cropping options to suit your design and usability constraints. With an additional 600 objects now supported, you can spend less time focusing your efforts on image cropping and more time looking at the bigger picture.<\/p>\n

The Simple Crop<\/h2>\n

Let\u2019s say your website allows users to upload photos of cars that they want to sell. You know nothing about the content of the photos, except that they are expected to contain cars. You want to save time for your users by automatically positioning the cars front and center, even if you will be displaying their photos in a different aspect ratio and size to the originally uploaded image. The simplest way to do this is to tell Cloudinary to keep only the car in the crop by using the crop<\/code> parameter and setting the gravity to car<\/code> (c_crop,g_car<\/code> in URLs). You don\u2019t even need to specify any dimensions:<\/p>\n

\n Loading code examples<\/span>\n <\/cld-code-widget><\/p>\n

\n<\/div>\n
\n\n\"Original<\/a>\nOriginal<\/b>\n<\/span>\n\n\"Image<\/a>\nc_crop,g_car<\/b>\n<\/span>\n<\/div>\n
\n<\/div> \n

You\u2019ll notice that the crop is very exact, so to give the car a bit of room to breathe, you may want to zoom out slightly: z_0.9<\/code>. And you\u2019ll probably want to limit the dimensions of the image too, so having kept only the car, you can then scale the image, say to a width of 200 pixels: c_scale,w_200<\/code>. Here\u2019s the full URL:<\/p>\n

\n Loading code examples<\/span>\n <\/cld-code-widget>\"Zoomed<\/a><\/p>\n

You\u2019ve now got a reusable transformation that can be applied to any car picture (if there\u2019s no car in the picture, the image won\u2019t be cropped, only resized):<\/p>\n

\n<\/div>\n
\n\n\n\n\n\n
\nOriginal (scaled to fit this table)<\/b>\n<\/td>\n\nKeep only the car (with a slight zoom out)
<\/b>c_crop,g_car,z_0.9\/
c_scale,w_200<\/code>\n<\/td>\n<\/tr>\n
\n\"Original<\/a>\n<\/td>\n\n\"Image<\/a>\n<\/td>\n<\/tr>\n
\n\"Original<\/a>\n<\/td>\n\n\"Image<\/a>\n<\/td>\n<\/tr>\n
\n\"Original<\/a>\n<\/td>\n\n\"Image<\/a>\n<\/td>\n<\/tr>\n
\n\"Original<\/a>\n<\/td>\n\n\"Image<\/a>\n<\/td>\n<\/tr>\n<\/table>\n<\/div>\n
\n<\/div> \n

One Size Fits All<\/h2>\n

If your website design calls for all images in a particular page or layout to be exactly the same size, then you\u2019ll want to set the aspect ratio too (together with the width we\u2019ve already added). You need to decide if you\u2019re happy for more of the original image to be shown, or if you still only want to see the chosen object.<\/p>\n

If you don\u2019t mind more of the original image being in the picture, then set the aspect ratio (ar<\/code>) in the c_crop<\/code> component (c_crop,g_car,z_0.9,ar_1.5\/c_scale,w_200<\/code>). Otherwise, instead of c_scale<\/code>, you could use the padding crop, c_pad<\/code> and set the aspect ratio in that component (c_crop,g_car,z_0.9\/c_pad,w_200,ar_1.5<\/code>). This adds padding to the image to reach the required size. You can either choose the color<\/a> of the background padding yourself, or let Cloudinary decide automatically<\/a> (b_auto<\/code>).<\/p>\n

\n<\/div>\n
\n\n\n\n\n\n
\nShow more of the picture in the crop<\/b>\n
c_crop,g_car,z_0.9,ar_1.5\/
c_scale,w_200<\/code>\n<\/td>\n		\nKeep only the car (with a slight zoom out),
and add automatically colored padding<\/b>\n
c_crop,g_car,z_0.9\/
c_pad,w_200,ar_1.5,b_auto<\/code>\n<\/td>\n<\/tr>\n
\n\"Image<\/a>\n<\/td>\n\n\"Image<\/a>\n<\/td>\n<\/tr>\n
\n\"Image<\/a>\n<\/td>\n\n\"Image<\/a>\n<\/td>\n<\/tr>\n
\n\"Image<\/a>\n<\/td>\n\n\"Image<\/a>\n<\/td>\n<\/tr>\n
\n\"Image<\/a>\n<\/td>\n\n\"Image<\/a>\n<\/td>\n<\/tr>\n<\/table>\n<\/div>\n
\n<\/div>\n

Order of Precedence<\/h2>\n

Now imagine it\u2019s not just cars that you\u2019re interested in, but vans and motorbikes too. You can add all three of these objects to your URL, but you\u2019ll need to define an order of precedence in case a photo has more than one of these in it. If you\u2019d prefer to see vans rather than cars, and cars rather than motorbikes, then specify this order: g_van:car:motorbike<\/code>. Applying this to the image of the car and van above gives this result:<\/p>\n

\n Loading code examples<\/span>\n <\/cld-code-widget>\"Giving<\/a><\/p>\n

Changing the order, to give precedence to the car, gives the result we had before:<\/p>\n

\n Loading code examples<\/span>\n <\/cld-code-widget>\"Giving<\/a><\/p>\n

If there are no cars or vans present in the picture, but there is a motorbike, the motorbike will be the focus of the crop:<\/p>\n

\n Loading code examples<\/span>\n <\/cld-code-widget><\/p>\n

\n<\/div>\n
\n\n\"Original<\/a>\nOriginal<\/b>\n<\/span>\n\n\"Image<\/a>\nc_crop,g_car:van:motorbike<\/b>\n<\/span>\n<\/div>\n
\n<\/div> \n

Object Categories<\/h2>\n

If you don\u2019t care what type of vehicle is kept in the crop, specifying the category, vehicle<\/code>, rather than specific objects will keep the most prominent vehicle in the crop. In this case, it\u2019s the van that wins.<\/p>\n

\n Loading code examples<\/span>\n <\/cld-code-widget>\"Crop<\/a><\/p>\n

If you want to keep all vehicles in the crop then it\u2019s as simple as adding an \u2018s\u2019 (this works for specific objects too):<\/p>\n

\n Loading code examples<\/span>\n <\/cld-code-widget>\"Keep<\/a><\/p>\n

Currently, the list of supported categories and objects<\/a> comes from two models: Common Objects in Context<\/a> (COCO) and Google\u2019s Open Images Dataset<\/a> (OID). Included are a wide variety of objects, from home appliances to food, electronics to furniture, people to animals, sports balls to handbags\u2026. The list is ever-expanding – object models enabling you to specify fashion items and parts of the body are coming soon. Additionally, in the near future, you\u2019ll be able to specify custom object models that cater specifically to your needs.<\/p>\n

Tailor-Made For Different Experiences<\/h2>\n

We\u2019ve seen how to fit different images into the same space, so now we\u2019ll look at how to fit the same image into different spaces, while still keeping the desired object as the focus of attention.<\/p>\n

An e-commerce website typically requires a handful of image crops for the same product images (thumbnails, product listing, detail view, and zoom). If you want your images to be responsive to different viewport sizes, yet more variations are required. Factor in social channels, and the number of different crops required quickly multiplies.<\/p>\n

To ensure that in each situation, your product, be it a handbag, a shoe or a vase, remains fully in view is a huge undertaking, one which requires creative teams to manage different crops, very painstakingly, and often manually.<\/p>\n

To help address this pain point, Cloudinary\u2019s Object-Aware Cropping<\/a> add-on automatically ensures that the object of interest, your product, is front and center for any crop, no matter which channel or device it\u2019s being viewed on.<\/p>\n

Let\u2019s take a look at a product image of a handbag. Here\u2019s the original image:<\/p>\n

\"Product<\/a><\/p>\n

A square crop is required for a website product detail and here we can use the fill<\/code> cropping method to fill the specified dimensions. To ensure that the handbag is kept in the crop we can choose between object-based automatic gravity (g_auto:handbag<\/code>) and object-specific gravity (g_handbag<\/code>). Object-based automatic gravity takes the whole picture into account, weighted by the handbag, whereas object-specific gravity focuses on the handbag over anything else, so the two can give different results. In this case, automatic gravity gives a slightly better crop because more of the hand, and less of the coat on the right is shown. Compare the two:<\/p>\n

\n<\/div>\n
\n\n\"Auto-gravity<\/a>\nc_fill,g_auto:handbag,h_800,w_800<\/b>\n<\/span>\n\n\"Object<\/a>\nc_fill,g_handbag,h_800,w_800<\/b>\n<\/span>\n<\/div>\n
\n<\/div> \n

Thumbnail crops are another option, and work well to scale down an image to fit the dimensions specified. Here\u2019s an example for a Pinterest board crop:<\/p>\n

\n Loading code examples<\/span>\n <\/cld-code-widget>\"Thumbnail<\/a><\/p>\n

This video demonstrates how different cropping methods and dimensions result in various crops of the same photo, all giving priority to the handbag:<\/p>\n

\n
\n

\n<\/video><\/p>\n<\/div>\n<\/center> \n\n