Client Hints and Responsive Images – Chrome 67 Changes

• Chrome 35 introduced Client Hints, a tool for simplifying Responsive Images.
• Cloudinary quickly deployed a number of Client-Hints-powered features:
  • Using w_auto, you can fit images to their layout width. Since the feature’s debut, it has delivered a median byte-size savings of 42% (51 KB) compared to the baseline.
  • With dpr_auto, you can enhance images to crisp, high-DPR images for the 60% of users with high-DPR displays (65% of desktop; 95% of mobile) while ensuring that the default is still a 1x, baseline image.
  • We’ve made q_auto aware of the Save-Data hint, saving even more bytes.
• A month ago, concerns were raised about Client Hints’ potential for tracking users across websites. As a result, desktop Chrome 67 /temporarily/ stopped sending Client Hints to third parties and subdomains, such as res.cloudinary.com and images.example.com.
• We were able to provide some feedback to the Chrome team which ensured that Cloudinary’s Client Hints-powered features still work on mobile Chrome. Desktop Chrome will work again as soon as the Feature Policy spec work and implementation are complete.

In 2015, Chrome 35 added support for Client Hints. Client Hints are awesome! With Client Hints, you can simplify the HTML required for Responsive Images, ensure that images are crisp on high-DPR displays, and improve #webperf. At the time of writing, Chrome 35+ is the only browser to have adopted this standard. Cloudinary has introduced support with the w_auto, dpr_auto, and q_auto parameters.

To refresh your memory, the four primary Client Hints are:

• Viewport-Width: This is the width of the browser window in CSS pixels. On mobile devices, this width is almost always 100% of the screen size, usually around 320 pixels (CSS).
• DPR: This is the device’s pixel ratio. For mobile devices, this ratio is usually between 2-4. Increasingly, desktop displays also have a DPR of 2.
• Width: This is the actual width—in device pixels—of the image as shown on screen. Note: This width is sent only if you also add the <img sizes> attribute.
• Save-Data: This flag, particularly useful on cellular, indicates that a server should help reduce data. (For completeness, this hint has moved to the NetInfo spec.)

w_auto typically reduces image bytes by 42%!

w_auto ensures that end-users receive image resources that are sized to fit the image’s layout size. Usually, that means scaling down an image, which reduces image bytes and makes for a faster experience.

For example, if you had a thumbnail image that fills only one-third of the screen, you could use the following HTML. Note the <img sizes> attribute.

<meta http-equiv="Accept-CH" content="DPR, Viewport-Width, Width">

<img src="https://res.cloudinary.com/demo/image/upload/w_auto/sample.jpg" sizes="33vw">

Users without Client Hints (eg: those who are on Firefox or Safari) do not get a resized image. To accommodate them, most developers set a fallback size, such as w_auto:100:500. Yet, if you compare the bytes sent by Chrome with Client-Hints to other (“simple-brain”) browsers, you can see that the median user downloads 42% less bytes, or 51KB per image, a significant savings.

Of course, different images see different savings. Some images are smaller by default; others are larger. Above is a Cumulative Distribution Function (CDF) of end-user experiences. On the y-axis is the percentile of user requests (a popular image is counted more times than an unpopular image). On the x-axis is the percent of bytes saved and the amount of bytes that Client Hints users saved per image as compared to non-Client Hints users.

As you can see, with Client Hints you can dramatically reduce the byte size per image. Responsive Images matter for performance!

60% of users have high-DPR screens

With the introduction of Retina displays on the iPhone 4, mobile and desktop manufacturers have increased physical-pixel density for a high-quality experience. To keep web pages from appearing really small on those devices, the number of logical, CSS pixels that are presented to web developers have remained constant (usually around 320 CSS pixels). However, you can multiply those CSS pixels by the device pixel ratio (DPR) to get the number of actual, physical pixels.

Currently, approximately 60% of all users have high-DPR displays. Ironically, it is mostly mobile devices that have those fantastic displays: 95% of Android devices compared to 65% of desktops.

Most content on the web is designed for DPR: 1 and, therefore, must be ‘“up-sampled”’ to fill in the extra pixels on the display. The result is a subpar experience, which might not matter for icons and logos. However, for hero images, marketing campaigns, product images, art, and such, you would want to enhance the experience by matching the DPR of the device.

Of course, you wouldn’t want to send a DPR: 3 image unnecessarily to a 1x device since it would result in unnecessary bytes. By adding the dpr_auto parameter, you can automatically enhance the user experience based on the device:

<meta http-equiv="Accept-CH" content="DPR, Viewport-Width, Width">
<img src="https://res.cloudinary.com/demo/image/upload/dpr_auto/sample.jpg" >

During a routine review, Google’s privacy team raised the concern that Client Hints could be used for fingerprinting users. Specifically, a person could be identified between websites without cookies but by correlating their screen size with the Viewport-Width and DPR attributes. That is possible because Client Hints, once enabled on, say, www.example.com, are sent not only to the www.example.com domain, but also to subdomains and third parties, such as images.example.com and evil.zombies.com. If multiple, unrelated web pages enable Client Hints and load sub-resources from evil.zombies.com, evil.zombies.com might be able to use those hints in their evil quest to uniquely identify some users, “fingerprinting” and tracking them as they move across the web.

Given that the viewport size of a Pixel 2 is the same for every device, you might wonder how one could use Client Hints to uniquely identify an individual. Also, how could request details like Viewport-Width: 320, DPR: 2, and Width: 500 be used for fingerprinting?

Good questions. Client Hints don’t reveal personal information. Nonetheless, in some situations – especially on desktops, where Viewport-Width varies wildly – those values could be combined with other details, e.g., your IP address, to make you measurably more unique than you already were. For example, if you don’t have your browser window set to full screen, you probably have a slightly different Viewport-Width value than that of most other users. By combining that browser-width with, say, your Starbucks WiFi IP address, evil.zombies.com could passively identify and fingerprint you you across multiple unrelated web pages.

We understood this concern in principle, but wanted to see what it looked like, in practice. So we looked at the existing values that we were receiving for Viewport-Width and found that 99% users fall into 40 standard viewport widths. However, the long tail of viewport-widths is very long – there are thousands of different values.

(Note that most of the 2.6% of “Other” Android values are probably “desktop” ChromeBooks.)

Out of an abundance of caution, the Chrome team decided to take action and disabled Client Hints for all cross-origin domains. So even if www.example.com has enabled Client Hints, desktop Chrome is not currently sending them to images.example.com or res.cloudinary.com.

Fortunately, we were able to share the benefits that our customers see with Client Hints, and make the case that hints from mobile devices should still be sent. The significant benefits of our Client Hints features mostly go to mobile users, and the privacy concerns are largely about desktop users. The Chrome team agreed, and decided to leave mobile/Android Chrome unchanged at this time).

The current plan, endorsed by both Google and the W3C, is to deliver fine-grain privacy controls to developers with the Feature Policy API. When that is in place in Chrome, desktop Client Hints for cross-origins will be re-enabled – and Cloudinary’s features will once again work for everyone.

One other change: the spec now requires TLS for the browser to send Client Hints. Fortunately, 80% of users are already benefiting from TLS. If you are in the 20% category, what’s holding you back from adopting TLS?

Despite the temporary service interruption, you, Cloudinary customer, can still realize the value of Client Hints. Here is a quick action plan for you:

1. Move to TLS. Change all of your links to Cloudinary to https://res.cloudinary.com.
2. At a minimum, enable (or keep using!) w_auto and q_auto. They will still work for mobile Chrome users, who generally benefit the most from these features.
3. For images that take up significantly less than the full viewport width, be sure to add an <img sizes>.
4. While we wait for Feature Policy, consider either using Cloudinary’s JavaScript SDK, – which can make w_auto and dpr_auto URLs work again (for a price paid in performance) – or implementing a markup-based responsive images solution (for a price paid in complexity).

If total bytes are more important to your users than speed, our JavaScript SDK can be a great stopgap solution, requiring minimal markup changes. But whereas Client Hints (and srcset) work with browsers’ preloaders, our JavaScript SDK must work around it, intentionally delaying image loads until after it knows each image’s layout size. So even though it will save kilobytes, using Javascript for responsive images costs seconds. Consider your/your users needs carefully.

We love Client Hints because they produce crisp, high-quality user experiences while also saving bytes! The recent changes in Chrome are a small setback but a precautionary compromise to balance user privacy (lessons, we have learned the hard way from companies like Cambridge Analytica). Cloudinary is active in the W3C standards working groups and is working to re-enable fully-functional Client Hints as soon as possible.

Watch for the upcoming release of the Feature Policy API, which will afford developers greater control over Client Hints — and much else besides. As always, feel free to join the discussion with your opinions or concerns about image standards.

In a follow-up post, we’ll share more insights on end-users of media content and best practices for using Client Hints. Just one teaser:

Stay tuned!

---

• Responsive images with Cloudinary
• Auto-Crop Images for Responsive Designs and Improved Image Quality
• Responsive Images: The srcset and sizes HTML Image Attributes
• Make All Images on Your Website Responsive in 3 Easy Steps
• Best Practices for Responsive Web Design