Cloudinary Blog
Automatic image scale and resolution with Client Hints

I'll start by giving it to you straight:

As part of the recent "auto–everything" launch, we introduced two new transformation parameters – dpr_auto and w_auto, which pair the DPR and Width Client Hints with Cloudinary’s existing image resizing and delivery infrastructure, in order to serve up simple, automatic responsive images.

If you’ve spent any time wrestling images into responsive layouts you know that those words — “simple”, “automatic”, and “responsive images” — are rarely found in honest proximity. So, take a quick peek at the demo and allow me give you a glimpse into the future of images on the web.

Client Hints live demo

Mo' resources, mo' problems

A cluster of matroyshka heads

Along with media queries and fluid grids, “flexible images” are one of the tentpoles of Ethan Marcotte’s Responsive Web Design. And, as Ethan pointed out in the seminal first edition of his book, achieving visually flexible images is as easy as:

img { max-width: 100%; }

For that rule to work though, the image resources backing it up must be big enough to fill large viewports and high-resolution displays. Shoving single-src images into responsive layouts means sending gigantic resources to everyone. Which is a performance disaster.

And so, along came a suite of new HTML features which allow images to adapt, ensuring that each user gets a resource that is tailored to their particular context. These features implement a key idea: authors should be able to mark up multiple, alternate resources. These features let us transform this bit of ham-fisted responsiveness:

<img src="enormous.jpg" 
     alt="Worst-case performance scenarios for all" />

…into an adaptable, multi-headed Hydra:

<img sizes="100vw"
     srcset="tiny.jpg      320w,
             small.jpg     512w,
             medium.jpg    640w,
             large.jpg    1024w,
             huge.jpg     1280w,
             enormous.jpg 2048w"
     src="fallback.jpg" 
     alt="To each according to his ability" />

But this adaptability comes at a price, paid in complexity. Generating, managing, and marking-up two or three (or six) “multiple, alternate” resources for every image can be both tedious and fiendishly complex. Services like Cloudinary can automate the resource generation side of the equation. But what to do about all of that markup?

Considering JavaScript

One answer to that question — one that we have embraced and implemented — is: use JavaScript.

Our last example offered the browser an array of options, allowing it to pick the best resource for the current context. A smart bit of JavaScript can directly measure the browsing context, and — when paired with on-the-fly, server-side resizing — request a single, perfectly-sized resource every time, with little or no extra markup required.

But... There are some problems with this approach. First, it requires setting up some JavaScript infrastructure, which is a complexity of a different flavor. Second, we’re inserting JavaScript between users and our pages’ core content, which can be tricky to do well. Worst of all, though — as it turns out — the smart people that make browsers for a living have noticed that images account for 64% of the web’s bytes, and they have gone to Herculean lengths to get those bytes across the wire as quickly as possible. Their main tool in that quest: the speculative pre-parser. In a nutshell, it tries to kick off as many image loads as possible[1] before a page’s HTML has even finished parsing. This results in an enormous 20% speed boost, and an intractable problem, when it comes to responsive images.

You see, for a JavaScript to load made-to-measure image resources, it must first measure the page — which means waiting until page layout is complete. This happens long after the pre-parser has raced through the HTML, kicking off image loads with wild abandon. This means that if we wish to use JavaScript to load responsive images, we face a choice. We must either:

  1. let the pre-parser do its thing, and, after a lengthy delay, set our Javascript loose to double-load all of our images, or,
  2. sabotage the pre-parser by authoring invalid, src-less <img>s so that our JavaScript can, after that same delay, start loading our pages’ images last, instead of first.

Both options present significant compromises; both set us back from achieving our over-arching goal, which is not, after all, simply to load appropriately-sized images. Our larger goal is performance.

Enter Client Hints

A small Matroyshka head popping out of a bunch of nested bottoms

So, where does that leave us? Performant, flexible images are inherently complex; marking up multiple, alternate resources can get verbose and complicated; trying to solve the problem with Javascript is fraught with compromise. What other tools do we have at our disposal? Where else can we shift the burden of this complexity?

The idea of Client Hints is: let’s put it on the server! If you’re a front-end developer, Client Hints means making responsive images somebody else’s problem. Cloudinary’s pitch to you, front-end-developer, is: make responsive images our problem! Leverage our Automatic Width and Automatic DPR features and we’ll handle this stuff so that you don’t have to.

Automatic DPR

Here’s a simple, live example:

Ruby:
cl_image_tag("bike.jpg", :width=>512, :dpr=>"auto", :crop=>"scale", :use_root_path=>true, :client_hints=>true)
PHP:
cl_image_tag("bike.jpg", array("width"=>512, "dpr"=>"auto", "crop"=>"scale", "use_root_path"=>true, "client_hints"=>true))
Python:
CloudinaryImage("bike.jpg").image(width=512, dpr="auto", crop="scale", use_root_path=True, client_hints=True)
Node.js:
cloudinary.image("bike.jpg", {width: 512, dpr: "auto", crop: "scale", use_root_path: true, client_hints: true})
Java:
cloudinary.url().transformation(new Transformation().width(512).dpr("auto").crop("scale")).clientHints(true).useRootPath(true).imageTag("bike.jpg")
JS:
cl.imageTag('bike.jpg', {width: 512, dpr: "auto", crop: "scale", use_root_path: true, client_hints: true}).toHtml();
jQuery:
$.cloudinary.image("bike.jpg", {width: 512, dpr: "auto", crop: "scale", use_root_path: true, client_hints: true})
React:
<Image publicId="bike.jpg" use_root_path="true" client_hints="true">
  <Transformation width="512" dpr="auto" crop="scale" />
</Image>
Angular:
<cl-image public-id="bike.jpg" use_root_path="true" client_hints="true">
  <cl-transformation width="512" dpr="auto" crop="scale">
  </cl-transformation>
</cl-image>
.Net:
cloudinary.Api.UrlImgUp.Transform(new Transformation().Width(512).Dpr("auto").Crop("scale")).UseRootPath(true).ClientHints(true).BuildImageTag("bike.jpg")
Android:
MediaManager.get().url().transformation(new Transformation().width(512).dpr("auto").crop("scale")).clientHints(true).useRootPath(true).generate("bike.jpg")
iOS:
let url = cloudinary?.createUrl().setUseRootPath( true).setTransformation(CLDTransformation().setWidth(512).setDpr("auto").setCrop("scale")).generate("bike.jpg")
imageView.cldSetImage(url!, cloudinary: cloudinary!)
Live Client-Hints responsive image example

The above URL delivers different resources to different users, depending on their context. In browsers that support Client Hints (at the moment, that’s mostly just Chrome), 1x devices will receive 1x resources; 2x screens will receive 2x resources. The above URL responds to varying DPRs automatically.

How? There’s a little tag in the <head> of this page which looks like this:

<meta http-equiv="Accept-CH" content="DPR">

That tells the browser to tack an extra HTTP header named DPR onto all of its subsequent requests, advertising the current devicePixelRatio. Those DPR HTTP headers are Client Hints. And smart servers can use them to tailor custom, smart responses. And so, when we opt-in to sending DPR Hints with a little meta magic, and add dpr_auto to our URL, we’re able to deliver different resources to different users depending on the density of their display.

Why else might we want to vary the scale of a resource in response to a fact about the browsing context? I’ll tell you why: to fit flexible, responsive layouts!

Automatic Width

Just like dpr_auto, w_auto sends different users different resources from the same URL. But whereas dpr_auto scales images to fit different display densities, w_auto scales images to fit variable layout widths. In its simplest form, w_auto looks like this:

<meta http-equiv="Accept-CH" content="DPR, Width">

[…snip…]

<img sizes="100vw"
     src="https://res.cloudinary.com/demo/w_auto/bike.jpg" 
     alt="Smiling girl with a bike." />

What have we done here?

First, we’ve used our meta tag to opt the browser into sending another Client Hint along with its requests: Width.

Next, note that our img includes a sizes attribute. This attribute tells the browser the layout width of the image [2] The browser then broadcasts that width to the server, via the Width hint. No sizes, no Width. No Width? w_auto does nothing. So (besides the opt-in meta tag), this is the one additional thing that w_auto asks of you: for it to work, you’ll have to stick sizes on your imgs.

Finally, note that I didn’t include dpr_auto in the URL. I didn’t have to! If we opt into sending both DPR and Width Hints in the meta tag up top, w_auto images will be scaled based on both of them.

So — that’s w_auto at its simplest; in Chrome, it just works. But, especially if you’re using the feature in production, you should dive a little deeper.

Advanced w_auto usage

Now the matroyshkas are wearing glasses

w_auto can take two optional parameters:

Ruby:
cl_image_tag("bike.jpg", :width=>"auto:100:400", :crop=>"scale", :use_root_path=>true, :client_hints=>true)
PHP:
cl_image_tag("bike.jpg", array("width"=>"auto:100:400", "crop"=>"scale", "use_root_path"=>true, "client_hints"=>true))
Python:
CloudinaryImage("bike.jpg").image(width="auto:100:400", crop="scale", use_root_path=True, client_hints=True)
Node.js:
cloudinary.image("bike.jpg", {width: "auto:100:400", crop: "scale", use_root_path: true, client_hints: true})
Java:
cloudinary.url().transformation(new Transformation().width("auto:100:400").crop("scale")).clientHints(true).useRootPath(true).imageTag("bike.jpg")
JS:
cl.imageTag('bike.jpg', {width: "auto:100:400", crop: "scale", use_root_path: true, client_hints: true}).toHtml();
jQuery:
$.cloudinary.image("bike.jpg", {width: "auto:100:400", crop: "scale", use_root_path: true, client_hints: true})
React:
<Image publicId="bike.jpg" use_root_path="true" client_hints="true">
  <Transformation width="auto:100:400" crop="scale" />
</Image>
Angular:
<cl-image public-id="bike.jpg" use_root_path="true" client_hints="true">
  <cl-transformation width="auto:100:400" crop="scale">
  </cl-transformation>
</cl-image>
.Net:
cloudinary.Api.UrlImgUp.Transform(new Transformation().Width("auto:100:400").Crop("scale")).UseRootPath(true).ClientHints(true).BuildImageTag("bike.jpg")
Android:
MediaManager.get().url().transformation(new Transformation().width("auto:100:400").crop("scale")).clientHints(true).useRootPath(true).generate("bike.jpg")
iOS:
let url = cloudinary?.createUrl().setUseRootPath( true).setTransformation(CLDTransformation().setWidth("auto:100:400").setCrop("scale")).generate("bike.jpg")
imageView.cldSetImage(url!, cloudinary: cloudinary!)

Responsive image with Client Hints example

The first (:100) tells Cloudinary how big the “jumps” between alternate resources should be. If you set this to :1, resources will be scaled to match the layout width exactly — which as it turns out, is a bad idea. To deliver perfectly-scaled resources, our servers must render and store hundreds, maybe thousands of alternate versions of your image, which is both computationally expensive and terrible for your pages’ performance. More, alternate resources means fewer cache hits, and a hit at any level — in the local browser cache, on the CDN, or server-side — will always be significantly faster than a miss. Thus, this first parameter allows you to limit the number of possible responses by defining a “rounding step” between them. For example, here, if the target width is 452 pixels, Cloudinary will round up to the nearest hundred and return a 500-pixel-wide resource.

The second parameter (:400) serves as a fallback Width, which will be used if the browser doesn’t send a Width Hint. In this example, browsers that don’t support Client Hints are served a 400-pixel-wide image. (400 is a “mobile-first” default. If you would rather serve a “desktop-first” default to non-supporting browsers, set this to something higher, like 1000). Word to the wise: without a Width or this fallback, w_auto images will be delivered at their original size. If you’re working with large originals — especially given Client Hints’ nascent support — this could be disastrous! You do not want to start sending 12 megapixel originals to Mobile Safari.

There is one more thing to know about these parameters; I’ve saved w_auto’s best trick for last.

Automatic breakpoints

Picking an arbitrary step-size using the first parameter can feel, well, confusingly arbitrary. Four years ago, Jason Grigsby pointed out that it makes more sense to think about these steps in terms of bytes, rather than pixels. We agreed, and built a tool that implements Jason’s idea — the Responsive Image Breakpoints Generator. We’ve now built that same logic into w_auto.

Our servers can look at your images, see how they compress, and determine byte-size-based gaps between images for you if you set the steps parameter to a special value: :breakpoints.

w_auto:breakpoints packages up a lot of smart thinking and builds on years’ worth of foundational effort to provide an astoundingly automatic responsive images solution that’s smart not only about images’ context — but also their particular content. Alas, :breakpoints requires a bit of gymnastics at the CDN layer, and in order to use it you’ll need to set up either a custom domain name or a private CDN distribution — features which are available only on Advanced or Enterprise/Custom Plans. Contact us if you’re interested in setting up :breakpoints on your account.

Speaking of the imperfect present…

Let’s talk about Client Hints’ browser support

A matroyshka has fallen over; a friend observes, concerned

Right now, Client Hints are only supported in Chrome and Opera. That means that dpr_auto and w_auto will work for approximately half of your users[3].

Other browsers have been slow to follow Google’s lead. Like every new web platform feature, Client Hints faces a chicken-and-egg problem. Vendors don’t want to implement features in browsers until they see a clearly-expressed demand; authors don’t want to use features on their pages until they enjoy almost universal support.

dpr_auto and w_auto are, among other things, our attempt to nudge Client Hints forward. We developed these features and are advocating for their adoption because we believe fully-automated, drop-dead simple, responsive, and performant images should not be the exception, but rather the norm.

So, should you use them?

If you’re starting a new project or retrofitting an old one, doing something about responsive images should be at the top of your to-do list. If you’re willing to put in the effort, a markup-based solution will get you the most browser support, but automating that markup can be complicated. JavaScript-based solutions are simpler, but operate within significant performance constraints.

Client Hints pave a third path, which is both simple and performant. And, despite browsers’ uneven Client Hints support, w_auto and dpr_auto provide a lot of value, right now.

The vast majority of websites aren’t doing anything about responsive images; most developers are still sending a single, wasteful, high-resolution resource to everyone. Implementing dpr_auto and w_auto requires next-to-no effort and delivers fully-responsive images to around half of your visitors without negatively impacting the other half. They provide tremendous performance gains now, and those gains will grow as browser support improves.

Client Hints are where the puck is going for images on the web. Sign up for a free account and start using them, today.

Further reading

2,000 words in and you still want more? I encourage you to dive into the following resources:

[1] Speculative pre-parsers initiate 43% of all image loads

[2] Sizes takes CSS lengths and optionally pairs them with media queries. A complete account of its abilities is out of scope for this article, but suffice to say, sizes (especially in combination with viewport relative units, the CSS calc function, and media queries), can express just about any dynamic layout width that you can dream up. Based on the information in sizes, browsers (and their speculative pre-parsers) can figure out how many px the image will occupy on the layout without knowing about, or needing to load, any other piece of the page.

[3] Depending on your audience, it could actually be a lot higher! For instance: 70% of our readers here on the Cloudinary blog use Chrome.

Recent Blog Posts

With automatic video subtitles, silence speaks volumes

The last time you scrolled through the feed on your favorite social site, chances are that some videos caught your attention, and chances are, they were playing silently.

On the other hand, what was your reaction the last time you opened a web page and a video unexpectedly began playing with sound? If you are anything like me, the first thing you did was to quickly hunt for the fastest way to pause the video, mute the sound, or close the page entirely, especially if you were in a public place at the time.

Read more
Impressed by WhatsApp Tech? Build WhatsApp Clone with Media Upload

With more than one billion people using WhatsApp, the platform is becoming a go-to for reliable and secure instant messaging. Having so many users means that data transfer processes must be optimized and scalable across all platforms. WhatsApp is touted for its ability to achieve significant media quality preservation when traversing the network from sender to receiver, and this is no easy feat to achieve.

Read more
New Google-powered add-on for auto video categories and tags

Due to significant growth of the web and improvements in network bandwidth, video is now a major source of information and entertainment shared over the internet. As a developer or asset manager, making corporate videos available for viewing, not to mention user-uploaded videos, means you also need a way to categorize them according to their content and make your video library searchable. Most systems end up organizing their video by metadata like the filename, or with user-generated tags (e.g., youtube). This sort of indexing method is subjective, inconsistent, time-consuming, incomplete and superficial.

Read more

iOS Developer Camp: The Dog House

By Shantini Vyas
iOS Developer Camp: The Dog House

Confession: I’m kind of addicted to hackathons. Ever since graduating from Coding Dojo earlier this year, I’ve been on the hunt for new places to expand my skills and meet new people in the tech space. iOS Developer Camp’s 10th Anniversary event bowled me over. Initially, because of its length. 48 hours? Yeesh. I had no idea that those 48 hours would change my life. But let’s first get a little backstory on my favorite topic: dogs.

Read more
GDPR: Cloudinary's take on the What, When, Why, and How

GDPR is a new regulation that deals with the way individuals' private information is handled. This regulation is going to have a deep effect on the entire internet industry. The fact that GDPR is a European regulation doesn't mean it's relevant only for European organizations. It requires protecting the data of any individual whose data is processed or stored in any way within European boundaries. As the reach of many companies is global, the requirement is actually relevant to a lot of companies worldwide.

Read more