Cloudinary Blog

Automatically art-directed responsive images

by Eric Portis
Art directed responsive images with picture and Cloudinary

This is a guest post by Eric Portis – a proud member of (and newsletter-writer for) the Responsive Issues Community Group. The RICG formulated, championed, and standardized the new HTML features presented in the article.

Previously, we saw how to mark up performant, adaptable <img>s using srcset and sizes in conjunction with Cloudinary’s image transformations.

How far can we push that notion of “adaptability"? Last time, we learned how to offer up an image at a range of different resolutions. This allowed us to send large resources to large screens and small resources to small ones. We used srcset and sizes to adapt for performance. But visually, our images remained fixed.

What if we could go a step further and adapt our image’s visual characteristics at breakpoints, just like we adapt our pages’ layouts? The term for this sort of adaptation is art direction. In this article, we’ll create an art-directed front page for our example site:

https://ericportis.com/etc/cloudinary/

The lead article’s preview image on this front page is huge, spanning the entire width of the page. On wide screens, in order to keep it from pushing the rest of the content “below the fold”, we need to crop it. Like this:

The rest of the image previews? They have the opposite problem. Left to simply shrink along with a narrowing viewport, they’d become too small to really make out. So on small screens, we want to “zoom in” on their subjects.

How can we achieve these things in markup? With a new element:

The picture element

The <picture> element was added to the HTML specification last year and has been implemented in every major desktop browser except for Safari (Safari support is right around the corner). <picture> gives us a way to supply alternate, visually distinct versions of an image, and switch them out at breakpoints.

The first thing to know about <picture> is that it’s a wrapper for <img>; think of <picture> as a kind of invisible, magical <span> which can feed its <img> alternate sources.

The second thing to know about it is that its markup pattern was adapted from <audio> and <video>. So alongside our <img>, we’ll pack our <picture> full of <source> elements.

Each <source> represents a visually distinct version of the image. We tell the browser which <source> to use and when, using media attributes.

<picture>
  <source media="(min-width: 800px)"
      sizes="100vw"
      srcset="cropped-for-wide-screens--large.jpg 1600w,
              cropped-for-wide-screens--small.jpg 800w" />
  <source media="(min-width: 600px)"
      sizes="100vw"
      srcset="full-image-for-standard-screens--large.jpg 1200w,
              full-image-for-standard-screens--small.jpg  600w" />
  <img
    src="zoomed-in-for-small-screens--small.jpg"
    srcset="zoomed-in-for-small-screens--large.jpg 800w,
            zoomed-in-for-small-screens--small.jpg 400w"
    alt />
</picture>

The first <source> element whose media attribute matches the current environment wins. The browser picks a resource out of that <source>’s srcset/sizes pair and feeds the picked resource to the <img>.

Et voila! An image that can change its appearance at breakpoints, as you can see in the examples above.

But dog-gone-it, we’ve done it again — by adding a new dimension of adaptability, we’ve multiplied the number of image resources we need to create and manage, making something that was once simple and static, dynamic and complex.

Cloudinary provides us with tools to manage that complexity.

Automatic cropping

A few months ago, I sat on on a stage at SmashingConf Freiburg, and Christian Heilmann asked me how, or if, one could automate the process of cropping in on the most important parts of an image. Stumped, I replied, “I don’t know, uh, something something neural networks?”

Right after my talk, Guy Podjarmy whisked me aside and showed me a few of Cloudinary’s auto-cropping features. I was amazed; now I get to show them to you!

First things first: in order to crop using Cloudinary, you need to specify a “crop mode”. We’ll start out by using the fill crop mode (c_fill in URLs), which works like background-fit: cover in CSS. The original image will be stretched or shrunk to cover the entirety of its new box, with any extra bits lopped off.

Let’s say we want to create a 100×100 square crop of our example image. Here’s how we’d do it:

Ruby:
cl_image_tag("on_the_phone.jpg", :width=>100, :height=>100, :crop=>"fill")
PHP:
cl_image_tag("on_the_phone.jpg", array("width"=>100, "height"=>100, "crop"=>"fill"))
Python:
CloudinaryImage("on_the_phone.jpg").image(width=100, height=100, crop="fill")
Node.js:
cloudinary.image("on_the_phone.jpg", {width: 100, height: 100, crop: "fill"})
Java:
cloudinary.url().transformation(new Transformation().width(100).height(100).crop("fill")).imageTag("on_the_phone.jpg")
jQuery:
$.cloudinary.image("on_the_phone.jpg", {width: 100, height: 100, crop: "fill"})
.Net:
cloudinary.Api.UrlImgUp.Transform(new Transformation().Width(100).Height(100).Crop("fill")).BuildImageTag("on_the_phone.jpg")
100x100 square image crop

How about a 640×360 version?

Ruby:
cl_image_tag("on_the_phone.jpg", :width=>640, :height=>360, :crop=>"fill")
PHP:
cl_image_tag("on_the_phone.jpg", array("width"=>640, "height"=>360, "crop"=>"fill"))
Python:
CloudinaryImage("on_the_phone.jpg").image(width=640, height=360, crop="fill")
Node.js:
cloudinary.image("on_the_phone.jpg", {width: 640, height: 360, crop: "fill"})
Java:
cloudinary.url().transformation(new Transformation().width(640).height(360).crop("fill")).imageTag("on_the_phone.jpg")
jQuery:
$.cloudinary.image("on_the_phone.jpg", {width: 640, height: 360, crop: "fill"})
.Net:
cloudinary.Api.UrlImgUp.Transform(new Transformation().Width(640).Height(360).Crop("fill")).BuildImageTag("on_the_phone.jpg")
640x360 cropped image

In addition to providing heights and widths, Cloudinary lets us supply aspect ratios, which can make our URLs a bit easier to read. This URL returns an image identical to the previous one:

Ruby:
cl_image_tag("on_the_phone.jpg", :aspect_ratio=>"16:9", :width=>640, :crop=>"fill")
PHP:
cl_image_tag("on_the_phone.jpg", array("aspect_ratio"=>"16:9", "width"=>640, "crop"=>"fill"))
Python:
CloudinaryImage("on_the_phone.jpg").image(aspect_ratio="16:9", width=640, crop="fill")
Node.js:
cloudinary.image("on_the_phone.jpg", {aspect_ratio: "16:9", width: 640, crop: "fill"})
Java:
cloudinary.url().transformation(new Transformation().aspectRatio("16:9").width(640).crop("fill")).imageTag("on_the_phone.jpg")
jQuery:
$.cloudinary.image("on_the_phone.jpg", {aspect_ratio: "16:9", width: 640, crop: "fill"})
.Net:
cloudinary.Api.UrlImgUp.Transform(new Transformation().AspectRatio("16:9").Width(640).Crop("fill")).BuildImageTag("on_the_phone.jpg")
Aspect ratio based image cropping

Ok, let’s try something really wide:

Ruby:
cl_image_tag("on_the_phone.jpg", :aspect_ratio=>"4:1", :width=>640, :crop=>"fill")
PHP:
cl_image_tag("on_the_phone.jpg", array("aspect_ratio"=>"4:1", "width"=>640, "crop"=>"fill"))
Python:
CloudinaryImage("on_the_phone.jpg").image(aspect_ratio="4:1", width=640, crop="fill")
Node.js:
cloudinary.image("on_the_phone.jpg", {aspect_ratio: "4:1", width: 640, crop: "fill"})
Java:
cloudinary.url().transformation(new Transformation().aspectRatio("4:1").width(640).crop("fill")).imageTag("on_the_phone.jpg")
jQuery:
$.cloudinary.image("on_the_phone.jpg", {aspect_ratio: "4:1", width: 640, crop: "fill"})
.Net:
cloudinary.Api.UrlImgUp.Transform(new Transformation().AspectRatio("4:1").Width(640).Crop("fill")).BuildImageTag("on_the_phone.jpg")
Wide aspect ratio cropping

This crop is… awkward. The president’s head is popping up from the bottom of the frame like a turnip.

By default, Cloudinary crops in on an image’s center. But what if we want to crop in on a different focal point? For that, we need to use Cloudinary’s “gravity” parameter. Our last crop chopped off the president’s body. Let’s aim lower, anchoring our crop to the bottom of the image:

Ruby:
cl_image_tag("on_the_phone.jpg", :aspect_ratio=>"4:1", :width=>640, :gravity=>"south", :crop=>"fill")
PHP:
cl_image_tag("on_the_phone.jpg", array("aspect_ratio"=>"4:1", "width"=>640, "gravity"=>"south", "crop"=>"fill"))
Python:
CloudinaryImage("on_the_phone.jpg").image(aspect_ratio="4:1", width=640, gravity="south", crop="fill")
Node.js:
cloudinary.image("on_the_phone.jpg", {aspect_ratio: "4:1", width: 640, gravity: "south", crop: "fill"})
Java:
cloudinary.url().transformation(new Transformation().aspectRatio("4:1").width(640).gravity("south").crop("fill")).imageTag("on_the_phone.jpg")
jQuery:
$.cloudinary.image("on_the_phone.jpg", {aspect_ratio: "4:1", width: 640, gravity: "south", crop: "fill"})
.Net:
cloudinary.Api.UrlImgUp.Transform(new Transformation().AspectRatio("4:1").Width(640).Gravity("south").Crop("fill")).BuildImageTag("on_the_phone.jpg")
Fixed width with 4:1 aspect ratio based cropping

Oops! Now we’ve chopped off his head! If only we could center the new, cropped image right on his face…

Ruby:
cl_image_tag("on_the_phone.jpg", :aspect_ratio=>"4:1", :width=>640, :gravity=>"face", :crop=>"fill")
PHP:
cl_image_tag("on_the_phone.jpg", array("aspect_ratio"=>"4:1", "width"=>640, "gravity"=>"face", "crop"=>"fill"))
Python:
CloudinaryImage("on_the_phone.jpg").image(aspect_ratio="4:1", width=640, gravity="face", crop="fill")
Node.js:
cloudinary.image("on_the_phone.jpg", {aspect_ratio: "4:1", width: 640, gravity: "face", crop: "fill"})
Java:
cloudinary.url().transformation(new Transformation().aspectRatio("4:1").width(640).gravity("face").crop("fill")).imageTag("on_the_phone.jpg")
jQuery:
$.cloudinary.image("on_the_phone.jpg", {aspect_ratio: "4:1", width: 640, gravity: "face", crop: "fill"})
.Net:
cloudinary.Api.UrlImgUp.Transform(new Transformation().AspectRatio("4:1").Width(640).Gravity("face").Crop("fill")).BuildImageTag("on_the_phone.jpg")
Face detection based cropping

g_face finds a face in the image and centers the crop on it, ensuring that if there is a person in our photo, they’ll remain front and center.

Putting it all to work

So! Now we’ve seen how to mark up visually adaptable images using <picture> and generate alternate crops automatically using Cloudinary. We have everything we need to art direct our example’s giant header image:

<picture>

  <!-- wide crop -->
  <source 
    media="(min-width: 600px)"
    srcset="http://res.cloudinary.com/eeeps/image/upload/c_fill,ar_2:1,g_face,f_auto,q_70,w_600/on_the_phone.jpg 600w,
            http://res.cloudinary.com/eeeps/image/upload/c_fill,ar_2:1,g_face,f_auto,q_70,w_1200/on_the_phone.jpg 1200w"
    sizes="100vw" />

  <!-- standard crop -->
  <img
    srcset="http://res.cloudinary.com/eeeps/image/upload/f_auto,q_70,w_400/on_the_phone.jpg 400w,
            http://res.cloudinary.com/eeeps/image/upload/f_auto,q_70,w_800/on_the_phone.jpg 800w"
    src="http://res.cloudinary.com/eeeps/image/upload/f_auto,q_70,w_400/on_the_phone.jpg"
    alt="President Obama on the phone in the Oval Office"
    sizes="100vw" />

</picture>

This complex-looking example should now make some sense. We start with an un-cropped <img> (which includes a srcset and sizes so that it’ll look good across resolutions), wrap it in a <picture>, and give it a <source> sibling. This <source> represents the cropped version of our image, and will only send a resource to the <img> when its media attribute (min-width: 600px) matches the current environment.

That chunk of code gets us this:

The hero image in our example is a bit more complex than this, with more breakpoints, more srcset resources, and a couple of additional Cloudinary tricks which we’ll cover in our next section. View-source-ing it upon completion of the article is left as an exercise to the reader.

Room to zoom

Let’s proceed to the thumbnails further down the page. Remember, they have the opposite problem — on small screens, they become too small. On small screens, we want to “zoom in” on their subjects.

In order to do so, we’ll use a new crop mode: c_thumb. When used with g_faces, c_thumb zooms all the way in on a face. Like this:

Ruby:
cl_image_tag("on_the_phone.jpg", :gravity=>"face", :aspect_ratio=>"1:1", :width=>256, :crop=>"thumb")
PHP:
cl_image_tag("on_the_phone.jpg", array("gravity"=>"face", "aspect_ratio"=>"1:1", "width"=>256, "crop"=>"thumb"))
Python:
CloudinaryImage("on_the_phone.jpg").image(gravity="face", aspect_ratio="1:1", width=256, crop="thumb")
Node.js:
cloudinary.image("on_the_phone.jpg", {gravity: "face", aspect_ratio: "1:1", width: 256, crop: "thumb"})
Java:
cloudinary.url().transformation(new Transformation().gravity("face").aspectRatio("1:1").width(256).crop("thumb")).imageTag("on_the_phone.jpg")
jQuery:
$.cloudinary.image("on_the_phone.jpg", {gravity: "face", aspect_ratio: "1:1", width: 256, crop: "thumb"})
.Net:
cloudinary.Api.UrlImgUp.Transform(new Transformation().Gravity("face").AspectRatio("1:1").Width(256).Crop("thumb")).BuildImageTag("on_the_phone.jpg")
Face detection based thumbnail

One gotcha with this technique: it will sometimes create a different crop, depending on the w specified. c_thumb will zoom in on the face as tightly as it can at the original image’s full resolution. If we specify a tiny w, it will happily scale the resulting, fully-zoomed face down:

Ruby:
cl_image_tag("on_the_phone.jpg", :gravity=>"face", :aspect_ratio=>"1:1", :width=>96, :crop=>"thumb")
PHP:
cl_image_tag("on_the_phone.jpg", array("gravity"=>"face", "aspect_ratio"=>"1:1", "width"=>96, "crop"=>"thumb"))
Python:
CloudinaryImage("on_the_phone.jpg").image(gravity="face", aspect_ratio="1:1", width=96, crop="thumb")
Node.js:
cloudinary.image("on_the_phone.jpg", {gravity: "face", aspect_ratio: "1:1", width: 96, crop: "thumb"})
Java:
cloudinary.url().transformation(new Transformation().gravity("face").aspectRatio("1:1").width(96).crop("thumb")).imageTag("on_the_phone.jpg")
jQuery:
$.cloudinary.image("on_the_phone.jpg", {gravity: "face", aspect_ratio: "1:1", width: 96, crop: "thumb"})
.Net:
cloudinary.Api.UrlImgUp.Transform(new Transformation().Gravity("face").AspectRatio("1:1").Width(96).Crop("thumb")).BuildImageTag("on_the_phone.jpg")
Face detection based thumbnail with custom aspect ratio and width

But, with a large w, instead of resizing the tightly-cropped-face up, it will pad it with the surrounding image at it’s full resolution:

Ruby:
cl_image_tag("on_the_phone.jpg", :gravity=>"face", :aspect_ratio=>"1:1", :width=>512, :crop=>"thumb")
PHP:
cl_image_tag("on_the_phone.jpg", array("gravity"=>"face", "aspect_ratio"=>"1:1", "width"=>512, "crop"=>"thumb"))
Python:
CloudinaryImage("on_the_phone.jpg").image(gravity="face", aspect_ratio="1:1", width=512, crop="thumb")
Node.js:
cloudinary.image("on_the_phone.jpg", {gravity: "face", aspect_ratio: "1:1", width: 512, crop: "thumb"})
Java:
cloudinary.url().transformation(new Transformation().gravity("face").aspectRatio("1:1").width(512).crop("thumb")).imageTag("on_the_phone.jpg")
jQuery:
$.cloudinary.image("on_the_phone.jpg", {gravity: "face", aspect_ratio: "1:1", width: 512, crop: "thumb"})
.Net:
cloudinary.Api.UrlImgUp.Transform(new Transformation().Gravity("face").AspectRatio("1:1").Width(512).Crop("thumb")).BuildImageTag("on_the_phone.jpg")
Face detection based thumbnail of larger width

Put another way: c_thumb will never upscale your images.

In order to generate consistent crops for arbitrary ranges of w values, we need to crop first, and resize second. We need to learn a new trick: chained transformations:

Ruby:
cl_image_tag("on_the_phone.jpg", :transformation=>[
  {:gravity=>"face", :aspect_ratio=>"1:1", :width=>512, :crop=>"thumb"},
  {:width=>96}
  ])
PHP:
cl_image_tag("on_the_phone.jpg", array("transformation"=>array(
  array("gravity"=>"face", "aspect_ratio"=>"1:1", "width"=>512, "crop"=>"thumb"),
  array("width"=>96)
  )))
Python:
CloudinaryImage("on_the_phone.jpg").image(transformation=[
  {"gravity": "face", "aspect_ratio": "1:1", "width": 512, "crop": "thumb"},
  {"width": 96}
  ])
Node.js:
cloudinary.image("on_the_phone.jpg", {transformation: [
  {gravity: "face", aspect_ratio: "1:1", width: 512, crop: "thumb"},
  {width: 96}
  ]})
Java:
cloudinary.url().transformation(new Transformation()
  .gravity("face").aspectRatio("1:1").width(512).crop("thumb").chain()
  .width(96)).imageTag("on_the_phone.jpg")
jQuery:
$.cloudinary.image("on_the_phone.jpg", {transformation: [
  {gravity: "face", aspect_ratio: "1:1", width: 512, crop: "thumb"},
  {width: 96}
  ]})
.Net:
cloudinary.Api.UrlImgUp.Transform(new Transformation()
  .Gravity("face").AspectRatio("1:1").Width(512).Crop("thumb").Chain()
  .Width(96)).BuildImageTag("on_the_phone.jpg")
Face detection based thumbnail with additional resizing

If we split two sets of transformations with a forward slash, Cloudinary will apply the first set of transformations and treat the resulting image as input to the second set. Neat.

Finally—what if we don’t want such a tight crop? To zoom back out from the subject’s face, we need to learn one more parameter: z. z lets us zoom in or out via a multiplier. Values less than one zoom out, and values greater than one zoom in. So, to zoom out so that the cropped face ends up at a quarter of its original, tightly-cropped size, we specify z_0.25.

Ruby:
cl_image_tag("on_the_phone.jpg", :transformation=>[
  {:gravity=>"face", :aspect_ratio=>"1:1", :zoom=>0.25, :width=>512, :crop=>"thumb"},
  {:width=>96}
  ])
PHP:
cl_image_tag("on_the_phone.jpg", array("transformation"=>array(
  array("gravity"=>"face", "aspect_ratio"=>"1:1", "zoom"=>0.25, "width"=>512, "crop"=>"thumb"),
  array("width"=>96)
  )))
Python:
CloudinaryImage("on_the_phone.jpg").image(transformation=[
  {"gravity": "face", "aspect_ratio": "1:1", "zoom": 0.25, "width": 512, "crop": "thumb"},
  {"width": 96}
  ])
Node.js:
cloudinary.image("on_the_phone.jpg", {transformation: [
  {gravity: "face", aspect_ratio: "1:1", zoom: 0.25, width: 512, crop: "thumb"},
  {width: 96}
  ]})
Java:
cloudinary.url().transformation(new Transformation()
  .gravity("face").aspectRatio("1:1").zoom(0.25).width(512).crop("thumb").chain()
  .width(96)).imageTag("on_the_phone.jpg")
jQuery:
$.cloudinary.image("on_the_phone.jpg", {transformation: [
  {gravity: "face", aspect_ratio: "1:1", zoom: 0.25, width: 512, crop: "thumb"},
  {width: 96}
  ]})
.Net:
cloudinary.Api.UrlImgUp.Transform(new Transformation()
  .Gravity("face").AspectRatio("1:1").Zoom(0.25).Width(512).Crop("thumb").Chain()
  .Width(96)).BuildImageTag("on_the_phone.jpg")
Zoomed-out face detection based thumbnail

And with that, we can smartly zoom our example page’s thumbnails on small screens, using <picture>, <source>, and Cloudinary:

<picture>

  <!-- full image -->
  <source
    media="(min-width: 600px)"
    srcset="http://res.cloudinary.com/eeeps/image/upload/f_auto,q_70,w_150/ronny.jpg 150w,
            http://res.cloudinary.com/eeeps/image/upload/f_auto,q_70,w_400/ronny.jpg 400w"
    sizes="calc(8em + 1vw)"
  />

  <!-- zoomed + square-cropped thumb for small screens -->
  <img 
    srcset="http://res.cloudinary.com/eeeps/image/upload/c_thumb,g_face,ar_1:1,z_0.25,w_180/f_auto,q_70,w_90/ronny.jpg 90w,
            http://res.cloudinary.com/eeeps/image/upload/c_thumb,g_face,ar_1:1,z_0.25,w_180/f_auto,q_70,w_180/ronny.jpg 180w"
    sizes="calc(4em + 3vw)"
    src="http://res.cloudinary.com/eeeps/image/upload/c_thumb,g_face,ar_1:1,z_0.25,w_180/f_auto,q_70,w_90/ronny.jpg"
    alt="President Obama speaking to Ronny Jackson"
  />

</picture>

A giant chunk of code like this can be intimidating; the trick is to look at it like a cake – multiple layers, each one building off of the last. Let’s break it down from the bottom.

We start, as always, with an <img> and some alt text, describing the image.

For browsers that can display images (and users that can see them), we include an <img src> on top of it, which contains the mobile-first default version of our image.

Browsers that understand srcset and sizes (which, these days, is just about all of them) will use these attributes, instead of the src, to select a resource to load, giving our image the ability to adapt to fit a range of viewport sizes and display densities.

Finally, we wrap our <img> in a <picture> and give it a <source> sibling, which will, in supporting browsers and on larger screens, allow the image to visually adapt, zooming out at a breakpoint when the viewport is sufficiently large.

Put it all together, and in a modern browser, you get this:

So there you have it – visually-adaptive, art-directed images using <picture>, <source>, and Cloudinary. Art direction opens up new frontiers in responsive design; Cloudinary’s on-the-fly face-detection, cropping, resizing, and optimization capabilities make it easy. So: go forth! And mark up performant, adaptable, progressively enhanced images for all.

Recent Blog Posts

Dynamic ZIP files generation with a single line of code

As a developer, you want to allow your users to download multiple files in a single click. An easy way to download multiple files and share them is to generate a ZIP file. When images are involved, you may also want to normalize the original images before including them in the ZIP file, by scaling them down to the same maximum resolution or converting them to the same format.

Read more
How mindbodygreen reduce costs & save development time

mindbodygreen is a digital media brand dedicated to health and wellness. The company’s website bring together leaders in the wellness world, best-selling authors, fitness experts, trusted healers, pioneering doctors, top chefs, celebrities and top-tier journalists to provide readers fresh, expert content and tools to help them live healthy, happy lives. The company was founded in 2009 and is headquartered in Brooklyn, N.Y. 
Read more

Why JPEG is like a photocopier

by Jon Sneyers
Why JPEG is like a photocopier (generation loss)

If you make a copy of a copy of a copy, the quality will deteriorate with every ‘generation’. This problem is called ‘generation loss’. It is not difficult to understand why this happens with actual copier machines. Scanning and printing are not perfect, being based on noisy sensors and physical paper and ink, and the resulting noise will tend to accumulate.

Read more
Apply conditions to dynamically manipulate images.

It's great to have the capability to manipulate images on the fly by using dynamic URLs to customize the images to fit the graphic design of your site or mobile application. However, what if you want to manipulate an image depending on a specific image characteristic (like its width or aspect ratio) or its contents (does it contain a face?). What you need is a way to apply a transformation to an image only if a specific condition is met. Take for example a situation where you have allocated space on your page for a user uploaded image with a width and height of 200 pixels. Furthermore, if the image contains a face you would like to zoom in and focus on the face itself, otherwise you would like to fit the entire image into the available space:

Read more
Moviepilot focus on core business and reduce costs
 
Moviepilot Inc. is a media company that hosts a website and social publishing platform designed to bring together movie and TV fans, and help studios trigger, fuel and amplify positive conversations about entertainment topics. The site invites fans to create content, such as writing articles and producing videos, which are then curated on the site and distributed to more than 30 million fans worldwide. The site boasts more than 20 million unique visitors per month, with 87 percent of visitors accessing the site on mobile devices. The company is headquartered in Venice Beach, Calif., and has an office Berlin, Germany, where it was originally founded.
Read more