Every image is unique, so are website visitors. In a perfect world, we would adapt images to be “just right” for all users, i.e., perfectly cropped with responsive dimensions, correct encoding settings, and optimal quality in the most suitable format.
See this example of a photo of a cat:
To have Cloudinary adapt and optimize that image to fit all devices and browsers through automation, add the parameters (in orange below) to the image URL:
The image on the left, a standard crop delivered as a standard JPEG of the original 2,000-px. height, is a large 537-KB file. In contrast, the image on the right is perfectly cropped and delivered as an optimized 170-KB WebP on Chrome, a bandwidth savings of 69 percent. You can save even more bandwidth by adopting one of our solutions for delivering responsive layouts, as described in the sections below.
Optimal delivery on an image-by-image and user-by-user basis is challenging. It requires considerable knowhow of the intricacies of file formats, vision algorithms, and browser behavior. The challenge takes on phenomenal proportions if you must scale countless images that are uploaded to your websites and mobile apps. Automation is the ultimate answer.
This is part of a series of articles about Auto Image Crop.
At its launch in 2012, Cloudinary, a complete image-management solution created by developers for developers, aimed at eliminating the R&D time required for managing images for web and mobile apps. A major milestone was reached in 2016 with the announcement of the state of Images Solved.
By way of background, developers must frequently create the perfect image crop that focuses on the most important region by first determining the optimal file format and encoder settings to represent the content with the minimum number of bytes. That means creating multiple variants of every image in different resolutions for responsive delivery on various devices.
Images Solved at Cloudinary automates the process of transforming images in a content-aware and context-aware manner. That is, for every high-resolution, user-uploaded image:
- Adapt it in real time to focus on its most important region.
- Dynamically select and incorporate the optimal quality and encoding settings.
- Responsively deliver the image to any device in any resolution or pixel density.
Further simplifying the life of developers are the following critical image-transformation and delivery tasks performed through automation by Cloudinary:
-
AI and Saliency based automated cropping:
g_auto -
Content-aware encoding:
q_auto -
Dynamic format selection:
f_auto -
Automatic responsive images:
w_autoanddpr_auto
Simply scaling images up or down on websites, especially responsive ones, usually doesn’t cut it. Oftentimes, you must crop images to fit device layouts, resolutions, and aspect ratios while highlighting the important regions within the images. Manually doing all that is impracticable, especially if your app includes massive user-generated content and a dynamic responsive layout. Also, cropping images the default way, i.e., focusing on the center of the image, can result in loss of important content.
You can crop photos with Cloudinary’s face-detection capability, but what if you want to focus on more than just the face or if the image subject is a product, a food item, or a pet? Enter Cloudinary’s automated cropping feature, which, through a combination of an AI-based algorithm and a saliency algorithm, automatically detects an image’s region of interest and then crops it on the fly with dynamic URLs to fit your site’s design and layout when viewed on any device.
For example, this uploaded photo must fit into three webpages: one in landscape mode (e.g., 800×200), one in portrait mode (e.g., 300×600), and one as a square (e.g., 300×300):
Standard center-cropping produces problematic results, with most of the cat hidden:
Setting the gravity image-cropping parameter to auto or adding g_auto to the URL generates optimal cropping, displaying the cat in full:
Note that Cloudinary generated the images above with the popular fill crop mode, retaining as much of the originals as feasible. However, to emphasize the object, websites sometimes prefer to display a zoomed-in thumbnail. Take these uploaded images:
To create a 300×270 thumbnail for each of the images, set gravity to face (g_face) in the URL to leverage Cloudinary’s face-detection feature. Here are the dynamically created, cloud-based versions:
The above thumbnails are passable. However, the blurry sofa in one overshadows the cat; the dog next to the lady in the top-right thumbnail is barely visible; and the red bike in the bottom one is completely hidden.
For a solution, leverage Cloudinary’s thumbnail-generation mode, which intelligently analyzes the image content and focuses on the region of interest. Setting the crop mode to thumb and gravity to auto (c_thumb,g_auto) produces the thumbnails below, effectively conveying the theme of each of the photos.
In addition, you can fine-tune the g_auto algorithm. For details, see the related documentation.
Have a try of Cloudinary’s interactive demo of smart, automated cropping.image_transformations#automatic_cropping).
A common concern for image edits is degradation of the visual quality, but quality loss is by far not always the case. In fact, precise adjustment of the compression level complemented by fine-tuning of the encoding settings can significantly reduce file size without any noticeable degradation. No single, optimal setting exists, though, because it depends on the compression algorithm and the image format and content.
Cloudinary automates the trade-off decision for file size versus quality on the fly with perceptual metrics and heuristics. The intelligent encoding algorithm analyzes the image to determine the best quality-compression level and the optimal encoding settings based on the image content. The result is a perceptually fine, size-minimized image.
To take advantage of Cloudinary’s automated capability for determining optimal encoding settings and the quality level for compression, set the quality transformation parameter to auto (q_auto in URLs).
For example, the following JPEG image, downscaled to 500-px.-wide, weighs 57.88 KB:
This image, generated with q_auto, passes muster, weighing only 39.21 KB, which is 32 percent lighter!
Some cases require further fine-tuning of the quality-selection algorithm. For instance, you can apply a more aggressive compression algorithm with the q_auto:low option, which results in a 22.51 KB image: –
For more details on the q_auto fine-tuning options, see the related documentation.
As effective and recommendable as Cloudinary’s automatic quality-algorithm is, you can go the manual route by setting an absolute value (between 1 and 100) for the quality level. See this example:
As additional reference, check out Cloudinary’s interactive demo on the automated quality-transformation feature and the related documentation.
Image formats can significantly affect page-load time and bandwidth, e.g., modern formats like WebP can reduce image sizes by 30 percent, leading to faster page loads, more site engagement, and higher conversion rates. A common developer task is to pick the optimal image formats for various scenarios based on the image content and the viewing device or browser, e.g., JPEG for captured photos or for faster loading, and PNG for illustrations or pictures with a transparent background. For Chrome, Safari, Internet Explorer, and Edge browsers, you must take into account additional logic for modern formats like WebP and JPEG-XR.
As simple as the requirements for browsers and format compatibility might be, manually computing the format logic for a large volume of images is complex and inefficient. Instead, turn to Cloudinary, which can dynamically select and convert images to the most efficient format according to the content and viewing browser, delivering, for example, images in WebP to Chrome, JPEG-XR to Internet Explorer or Edge, and JPEG-2000 for Safari.
In addition to transparent WebP delivery and JPEG-XR delivery, you can combine Cloudinary’s automated capabilities for selecting the best format, encoding images, and setting the optimal quality level. For example, with the q_auto,f_auto parameters, Cloudinary would encode the picture of the cat as a 15.67-KB WebP for Chrome, a JPEG-XR for Internet Explorer and Edge, a JPEG-2000 for Safari, and a JPEG for other browsers.
Combining q_auto and f_auto delivers a more sophisticated result than setting only one of those two parameters. A case in point: the Cloudinary algorithm might pick PNG or lossless WebP for images that contain drawings, and PNG8 or PNG24 if the quality algorithm finds it to be optimal for an ideal display and a small file size.
This URL dynamically generates a 500-px.-wide JPEG version of a drawing with the image-encoding and quality-selection parameter(q_auto without f_auto):
The above JPEG weighs 41 KB. A careful look would reveal that the lossy nature of JPEG produced a few undesirable artifacts. Adding both q_auto and f_auto to the URL causes the algorithm to encode the image with PNG8, generating an artifact-free image that weighs only 34.3 KB. (The only artifacts are from the original high-quality JPEG.)
For more details, see the related documentation.
Responsive designs require multiple versions of every image for a sharp display on all devices and at all resolutions, pixel densities, and orientations. Editing, generating, managing, and delivering numerous image versions is a daunting task. Not so with Cloudinary, which dynamically delivers images for responsive websites on Retina and regular displays by automating the image width and determining the DPR value based on the viewing device, display size, and layout.
Cloudinary comes to the rescue by determining the image width based on the browser’s viewport or layout width, subsequently computing the image’s optimal resolution for display on that device. Furthermore, Cloudinary can dynamically set image-specific breakpoints by determining how many image versions are required to balance the trade-off between optimal dimensions and bandwidth reduction.
All you need to do is upload a high-resolution version of your images to Cloudinary and make use of either JavaScript or Client Hints to have Cloudinary generate responsive images through automation.
Important: Due to privacy concerns on Client Hints’ potential for tracking users across websites, desktop Chrome 67 has temporarily stopped sending Client Hints, which still work on mobile Chrome, however.
After choosing the one of the above solutions, you can, through dynamic URLs, adapt images to fit the viewport, layout, and resolution on any device, ensuring a seamless display and optimal performance. To have Cloudinary choose and retrieve the images with best-fitting widths and DPR values, add to the dynamic URL the w and dpr parameters set to auto (dpr_auto,w_auto).
By default, Cloudinary rounds up the device’s DPR to an integer value (1.0, 2.0, 3.0, etc.), and the required width to the closest multiple of 100 px.
Note these two caveats, however:
- One image cannot accommodate all screen resolutions and devices.
- Generating an image for every pixel change in width is overkill, yet producing one for every 100-px. difference might be arbitrary.
Therefore, you can fine-tune breakpoints:
- With Cloudinary’s JavaScript SDK, you can pass a function to generate breakpoints. For the procedure, see the related documentation.
- Through Client Hints, you can leverage Cloudinary’s feature for intelligently generating responsive breakpoints. Here’s the related documentation.
In addition to the advanced image management techniques discussed here, it’s important to recognize the broader scope of smart cropping in the digital media landscape. This includes:
- Video Cropping: Particularly pertinent in social media contexts such as Facebook’s Creator Studio, allows for reframing and resizing the aspect ratios of videos. This tool is essential for content creators who wish to optimize their video content for different platforms and devices.
- Adobe’s AI and Machine Learning Tools: Adobe’s suite of tools utilizes AI and machine learning to automatically detect and crop the focal point of both images and videos. This highlights the growing trend of incorporating intelligent algorithms in various media editing tools beyond static images.
- Image and Screenshot Compacting: Another emerging technology is the automated compacting of images and screenshots. By identifying and eliminating redundant blank and insignificant areas, this tool streamlines images for more efficient storage and usage, a concept that extends the principles of smart cropping to image optimization.
- App-Based Video Cropping: The proliferation of mobile applications for video cropping, available on platforms like the App Store, allows users to manually crop videos frame by frame. This indicates the demand for more hands-on, customizable cropping tools in the mobile domain.
- Canva’s Photo Cropping: Tools like Canva offer instant photo cropping features, focusing on the most interesting parts of an image. This user-friendly approach to image editing caters to a wide range of users, from professional designers to social media enthusiasts.
These additional dimensions of smart cropping, encompassing both image and video content across various platforms and tools, illustrate the versatility and importance of this technology in today’s digital environment.
Related content: Read our guide to automatic image cropping and crop image css.
The four image-automation capabilities described in this post, which promise to be extremely useful for web developers, are available in all Cloudinary plans, including the free plan. Do try out the capabilities and let us know what you think. From my personal experience, once you start using them, you’ll be hard pressed to recall how you manually transformed images before.
- Currently, the parameter for responsive breakpoints (
w_auto:breakpoints) requires a custom domain name (CNAME) or private CDN distribution, which is available in Cloudinary’s Advanced and higher plans. This feature might also incur additional costs. - As intelligent as automatic algorithms are, they are not perfect, let alone that the quality of the result might be subjective. As an alternative, configure a manual override through the Cloudinary API or UI, i.e., override automated image-cropping with custom coordinates and automated computation of the quality level with a custom level. Such overrides are valuable input for future enhancements of our algorithms.
