{"id":21428,"date":"2016-10-20T12:43:25","date_gmt":"2016-10-20T12:43:25","guid":{"rendered":"http:\/\/push_button_art_direction_with_cloudinary_s_responsive_image_breakpoints_generator"},"modified":"2024-06-01T15:07:39","modified_gmt":"2024-06-01T22:07:39","slug":"push_button_art_direction_with_cloudinary_s_responsive_image_breakpoints_generator","status":"publish","type":"post","link":"https:\/\/cloudinary.com\/blog\/push_button_art_direction_with_cloudinary_s_responsive_image_breakpoints_generator","title":{"rendered":"Push-button Art Direction with Cloudinary\u2019s Responsive Image Breakpoints Generator"},"content":{"rendered":"

Note:<\/strong> this article was originally published in Smashing Magazine<\/a>.<\/em><\/p>\n

Four years ago, Jason Grigsby asked a surprisingly<\/em> difficult question: How do you pick responsive images breakpoints?<\/a> A year later, he had an answer<\/a>: ideally, we\u2019d set responsive image performance budgets to achieve \u201csensible jumps in file size\u201d. Cloudinary built a tool that implemented this idea<\/a>, and the response from the community was universal: \u201cGreat! Now \u2013 what else can it do?\u201d Today, we have an answer: art direction!<\/p>\n

Since its release earlier this year, the Generator<\/a> has been turning high-res originals into responsive <img><\/code>s with sensible srcset<\/code>s at the push of a button. Today, we\u2019re launching v2.0, which allows you to pair layout breakpoints with aspect ratios, and generate art-directed <picture><\/code> markup with smart-cropped image resources to match. Check it out<\/a>, and read on.<\/p>\n

Responsive image breakpoints: asked and answered<\/h2>\n

Why did we build this tool in the first place?<\/p>\n

\u201cResponsive images\u201d send different people different resources, each tailored to their particular context; a responsive image is an image that adapts<\/strong>. There are a number of different axes along which that adaptation can happen. Most of the time, most developers only need adaptive resolution<\/strong> \u2013 we want to send high resolution images to large viewports and\/or high-density displays, and lower-resolution images to everybody else. Jason\u2019s \u201cresponsive image breakpoints\u201d question concerns this sort of adaptation.<\/p>\n

When we\u2019re crafting images that adapt to fit variable resolutions, we need to generate a range of differently-sized resources. We need to pick a maximum resolution, a minimum resolution, and (here\u2019s the tricky bit) some sizes in between. The max and min can be figured out based on the page\u2019s layout and some reasonable assumptions about devices. But when developers began implementing responsive images, it wasn\u2019t at all clear how to size the in-betweens. Some people picked a fixed step size<\/strong> between image widths \u2013\n\"Rectangles Rectangles showing the relative dimensions of a group of srcset<\/code> resources that use a fixed-step-size strategy.<\/em><\/p>\n

others picked a fixed number of steps<\/strong>, and used it for every range \u2013<\/p>\n

\"Rectangles Rectangles showing the relative dimensions of three groups of srcset<\/code> resources that use a fixed-number-of-steps strategy.<\/em><\/p>\n

some people picked common display widths:<\/strong><\/p>\n

\"Rectangles Rectangles showing the relative dimensions of a group of srcset<\/code> resources scaled to common display widths.<\/em><\/p>\n

At the time, because I was lazy and didn\u2019t like managing many resources, I favored doubling:<\/strong><\/p>\n

\"Rectangles Rectangles showing the relative dimensions of a group of srcset<\/code> resources scaled using a doubling strategy.<\/em><\/p>\n

All of these strategies are essentially arbitrary. Jason thought there had to be a better way. And eventually realized that we shouldn\u2019t be thinking about these steps in terms of pixels<\/strong> at all. We should be aiming for \u201csensible jumps in file size\u201d; these steps should be defined in terms of bytes<\/strong><\/a>.<\/p>\n

For example, let\u2019s say we have the following two JPEGs:<\/p>\n

\"300<\/p>\n

300 pixels wide (41 KB)<\/em><\/p>\n

\"1200<\/p>\n

1200 pixels wide (336 KB)<\/em><\/p>\n

The biggest reason we don\u2019t want to send the 1200-pixel-wide resource to someone who only needs the small one isn\u2019t the extra pixels<\/strong>, it\u2019s the extra 295 KB of useless data. But different images compress differently; while a complex photograph like this might increase precipitously in byte-size with every increase in pixel-size, a simple logo might not add much weight at all. For instance, this 1000-pixel-wide PNG<\/a> is only 8 KB larger than a 200-pixel-wide version<\/a>.<\/p>\n

Sadly, there weren\u2019t any readily-useable tools to generate images at target byte-sizes. And ideally<\/strong>, you\u2019d want something that could generate whole ranges of responsive image resources for you \u2014 not just one at a time. Cloudinary built that tool<\/a>!<\/p>\n\"A\n

And released it as a free, open source<\/a> web app.<\/p>\n

But the people wanted more\u2026<\/p>\n

The next frontier? Automatic art-direction!<\/h2>\n

So: we built a solution to the the breakpoints problem. And in the process, built a tool that made generating resolution-adaptable<\/strong> images easy. Upload a high-resolution original; get back a fully-responsive <img><\/code> with sensible breakpoints and the resources to back it up.<\/p>\n

That basic workflow \u2013 upload an image, get back a responsive<\/strong> image \u2013 is appealing. We\u2019d been focusing on the breakpoints problem, but when we released our solution people were quick to ask \u2014 what else can it do?<\/p>\n

Remember when I said that resolution-based adaptation is what most developers need, most of the time? Sometimes, it\u2019s not enough. Sometimes, we want to adapt our images along an orthogonal axis: art direction<\/a>.<\/p>\n

Any time we alter our images visually<\/strong> to fit a different context, we\u2019re \u201cart directing\u201d. A resolution-adaptable image will look identical everywhere \u2013 it only resizes. An art-directed image changes in visually-noticeable ways. Most of the time, that means cropping, either to fit a new layout, or to keep the most important bits of the image legible when it\u2019s viewed at small physical sizes.<\/p>\n\"On \n

On small screens, we want to zoom in on the image\u2019s subject.<\/em><\/p>\n

People asked us for automatic art direction.<\/p>\n

Which is a hard problem! It requires knowing what the \u201cmost important\u201d parts of the image are. Bits and bytes are easy enough to program around; computer vision and fuzzy notions of \u201cimportance\u201d are something else entirely.<\/p>\n

For instance, given this image\u2026<\/p>\n

\n Loading code examples<\/span>\n <\/cld-code-widget>\"A<\/a><\/p>\n

\u2026a dumb algorithm might simply crop in on the center:<\/p>\n

\n Loading code examples<\/span>\n <\/cld-code-widget>\"The<\/a><\/p>\n

What you need is an algorithm that can somehow \u201csee\u201d the cat, and intelligently crop in on it.<\/p>\n

It took us a few months, but we built this, too, and packaged it up as a feature available to all Cloudinary users: g_auto<\/code><\/a>.<\/p>\n

Here\u2019s how it works: when you specify that you want to crop your image with \u201cautomatic gravity\u201d, the image is run through a series of tests, including edge-detection, face-detection, and visual uniqueness. These different criteria are then all used to generate a heat-map of the \u201cmost important\u201d parts of the image.<\/p>\n\"The\n

The master rolled-up heat map<\/em><\/a><\/p>\n

A frame with the new proportions is then rolled over the image, possible crops are scored, and a winner is chosen. Here\u2019s a visualization of the rolling frame algorithm (using a different source image<\/a>):<\/p>\n

\n

\n Loading code examples<\/span>\n <\/cld-code-widget>\"The<\/a><\/p>\n

Neat!<\/p>\n

It was immediately obvious that we could and should use g_auto<\/code>\u2019s smarts to add automatic art-direction to the Generator. After a few upgrades to the markup logic and some (surprisingly-tricky) UX decisions, we did it: version 2 of the tool \u2014 now with art-direction \u2014 is live<\/a>.<\/p>\n

Let\u2019s take a tour<\/h2>\n

How do you use the Responsive Image Breakpoints Generator?<\/p>\n

The workflow has been largely carried-over from the first version: upload an image (or pick one of our presets), and set your maximum\/minimum resolutions, a step size (in bytes!), and a maximum number of resources (alternatively, you can simply use our pretty-good-most-of-the-time defaults). Click \u201cgenerate\u201d, et voila!<\/strong> You\u2019ll get a visual representation of the resulting image\u2019s responsive breakpoints, some sample markup, and a big honkin\u2019 \u201cdownload images\u201d button.<\/p>\n\"Screenshot \n

Screenshot of the tool\u2019s inputs<\/em><\/p>\n\"Screenshot \n

Screenshot of the tool\u2019s outputs<\/em><\/p>\n

The new version has a new set of inputs, though, which enable art direction. They\u2019re off by default. Let\u2019s turn a couple of them on and regenerate, shall we?<\/p>\n\"Screenshot \n

Screenshot of the art-direction inputs, with \u201cDesktop\u201d and \u201cSmartphone\u201d selected<\/em><\/p>\n

The first output section is unchanged: it contains our \u201cdesktop\u201d (full) image, responsively-breakpointed to perfection. But below it there\u2019s a new section, which shows off our new, smartly-cropped image:<\/p>\n\"Screenshot \n

Screenshot of the \u201c1:1 Aspect Ratio\u201d section<\/em><\/p>\n

And below that<\/strong>, we now have all of the markup we need for an art-directed <picture><\/code> element that switches between the two crops at a layout breakpoint.<\/p>\n\"Screenshot \n

Screenshot of the markup section<\/em><\/p>\n

Finally, there\u2019s a live <picture><\/code> example, which shows you what all of that markup actually does.<\/strong><\/p>\n\"Screenshot \n

Screenshot of the \u201cLive Picture Element in Action\u201d section<\/em><\/p>\n

Lets circle back and look at the art direction inputs in a little more detail.<\/p>\n\"Screenshot \n

Screenshot of the art direction inputs, annotated to point out what each thing does<\/em><\/p>\n

Each big box maps to a device-type, and each device-type has been assigned a layout breakpoint. The text under the device-type name shows the specific media query that, when true, will trigger this crop.<\/p>\n

Below that, we can specify the aspect ratio we want to crop to on this device-type.<\/p>\n

Below that<\/strong>, we specify how wide the image will appear relative to the width of the viewport on this type of device. Will it take up the whole viewport (100%), or less than that? The tool uses this percentage to generate simple sizes<\/code><\/a> markup \u2013 which reflects\/specifies how large the image is on the layout. If you\u2019re using this code in production, you\u2019ll probably want to go back into the example markup and tailor these sizes<\/code> values to match your particular layout more precisely. But depending on your layout, rough estimates, input here, might be good enough.<\/p>\n

And there you have it: simple, push-button art direction.<\/p>\n

Automation<\/h2>\n

What if you want to work with more than one image at a time? If you\u2019re building entire sites, with hundreds, or thousands (or hundreds of thousands!) of images \u2013 especially if you\u2019re working with user-generated content \u2013 you\u2019ll want more than push-button ease; you\u2019ll need full automation. For that, there\u2019s Cloudinary\u2019s API, which you can use to call the smart-cropping and responsive image breakpoints functions that power the Generator, directly. With the API, you can create customized, optimized, and fully<\/strong>-automated responsive image workflows for projects of any shape or size.<\/p>\n

For instance, here\u2019s Ruby code that would upload an image to Cloudinary, smart-crop it to a 16:9 aspect ratio, and generate a set of downscaled-resources with sensible responsive image breakpoints:<\/p>\n

\tCloudinary<\/span>::Uploader.upload(\"sample.jpg\"<\/span>,\n\t    responsive_breakpoints<\/span>: { \n\t        create_derived<\/span>: true,\n\t        bytes_step: 20000<\/span>,\n\t        min_width: 200<\/span>,\n\t        max_width: 1000<\/span>,\n\t        transformation: {\n\t            crop: :fill,\n\t            aspect_ratio: \"16:9\"<\/span>,\n\t            gravity: :auto\n\t        }\n\t    }\n\t)\n<\/code><\/span>Code language:<\/span> CSS<\/span> (<\/span>css<\/span>)<\/span><\/small><\/pre>\n
Auto breakpoints URL<\/a><\/p>\n
There\u2019s a tremendous amount of smarts packed into that little URL!<\/p>\n
Final thoughts<\/h2>\n
But, back to the Generator. Now, it can do more than \u201cjust\u201d pick your image breakpoints \u2013 it can pick your art-directed crops too. And it will do all of the tedious resource-and-markup generation for you; upload one, high-res original, and get back all of the markup and down-scaled resources you need to include a scalable and<\/strong> art-directed image on your webpage.<\/p>\n
Have I mentioned that the Responsive Image Breakpoints Generator is free? And open source<\/a>? Give it a whirl<\/a>, and please, send us feedback. Who knows, maybe we\u2019ll be back again soon with version 3!<\/p>\n
Further Reading on Responsive Images<\/h2>\n
If you\u2019re only working on the front end, all of this functionality is available via URL parameters, too! Here\u2019s a Client-Hints<\/a>-and-smart-crop<\/a>-powered URL that does the same thing on download<\/strong> that the Ruby above does at upload<\/strong> \u2013 and<\/strong> delivers different, dynamically optimized resources to different devices, responsively:<\/p>\n