Cloudinary Blog

Jason Grigsby on Responsive Images: Where It All Started

Jason Grigsby on the Beginning of Responsive Images
 
Portland-based web developer Jason Grigsby co-founded Cloud Four. He’s a prolific writer and speaks about front-end development as thoughtfully as anyone in the business. He’s been writing about responsive images since there were responsive images to write about, and recently, he and I had a chance to sit down and talk about the past, present, and future of images on the web.
Here, in Part 1 (of a three-part series), we discuss how the industry slowly came to understand the surprisingly hard problems around “flexible media,” what’s been done to solve them, and the pain points and open questions that remain.
 
Eric Portis:  One of the interesting points I’ve heard you make — which is a nice entry point into all of this — is that images on the web have always been complicated. Can you give me a sense of what problems developers were facing in the early days of the web?  What did we have to worry about then, that we don’t have to worry about now?
 
Jason Grigsby: So basically, you’re starting off by asking me, okay grandpa—
 
EP: [Laughs]

JG: —Grandpa, tell me what it was like, I’m too young to understand. I see how you are! Well, in the earliest days, the web started without any images at all, and when we first started incorporating them, the problems were tough.  There were differences in color space between Macs and PCs, and we had to make sure that images were compressed well enough to download on connections that are much slower than we have now. Lynda Weinman, of Lynda.com, wrote books — I think it was five books — that were each several hundred pages long, all on images, and they were best sellers. In addition there were a number of specialized tools — I remember a piece of software called Debabelizer — that allowed us to handle images and GIF compression much better than Photoshop did.  It took awhile for Photoshop to add the ability to export for the web. 
 
Linda Wineman’s books
Linda Wineman’s Designing Web Graphics, Coloring Web Graphics, Preparing Web Graphics, and Deconstructing Web Graphics - 1,333 pages all about images!
 
So yes, I think there is a knee jerk reaction that a lot of people have when they start looking at responsive images. They think that the responsive images syntax is too complex and that it should be simple like images have always been. I reject that. Images are complex by their very nature. We’ve gotten good at automating some things and we’ll get better at automating others in the future. But to think that they’re not complex is to deceive yourself. They’ve always been complex.
 
Even after all of the time that I’ve spent on them, I still don’t feel like I understand the full complexity of images. And I think about them far too much.
 
EP: Let’s fast forward to 2010 or so, when developers started asking hard questions about images in responsive layouts.  You were right there at the beginning of those conversations, saying that, hey, something here is broken. What were the first problems that you and other developers ran up against? What were some of the early ideas floating around about how to solve those problems?
 
JG: Back in 2010, I wrote an article with the unfortunate title, “CSS Media Query for Mobile is Fool’s Gold.” I wrote it because about a year before – in 2009 – my company Cloud Four was brought in to consult on a site that was having a ton of performance problems. The site used media queries to do what we now call “responsive design,” before Ethan Marcotte coined that phrase.
 
We dug into what was going on, and found that the media queries were hiding assets. On this site, it was a Google map, which wasn’t necessary on small screens. But we discovered that the map was still loading – along with 47 other files, tiles and a bunch of other stuff – which made the page slow to a crawl.  
 
After Ethan wrote “Responsive Web Design”, I started seeing similar problems everywhere. There were pages where images were hidden but still downloading in the background. Similarly, images that were far too large were being shrunk down for small screens. As a result, you’d have a page that looked great, but it was 5MB in size.  
 
The analogy I make for this problem is that it’s akin to having a dinner party, but not having time to clean, so you shove everything into the closet. It may look like your house is clean, but you’ve really just hidden the problem and you’re ok if no one opens that door.  But on a website, even if it looks good, you can’t hide the performance impact of large assets.
 
About a year later my co-founder Lyza Gardner and I started writing “Head First Mobile Web,” and we wanted to have a chapter on “mobile first” responsive design. At the time people thought, ok, if we start with a minimal set of small assets and only load larger assets if they’re needed, we can solve the responsive image problem.  So I started looking at the techniques people that were using to do this, because I assumed that enough time had gone by — that by then somebody had to have a good solution. But I found that every single technique had problems.  They were all insufficient, even on high-profile sites. The Filament Group built the Boston Globe site with assumptions based on browser behavior that they saw in 2010, when they started working on the project.  But by 2011, when the site launched, that behavior had changed and their attempts at responsive images were no longer working. Experiences like that showed us that this needed to be fixed at a lower level. Things needed to change from a standards perspective. 
 
Boston Globe responsive website

The Filament Group’s landmark 2011 Boston Globe redesign proved that responsive design could be practical (and performant!) at scale. It also highlighted the “Responsive Images” problem – Filament Group developer Mat Marquis wrote prominently about the image problems that the Globe team encountered (and, later, founded/chaired the W3C Responsive Images Community Group, which solved them).

EP:  One thing I’ve found curious is that the push for responsive images really came from the bottom up — from developers like Cloud Four and The Filament Group — rather than from the top down. And initially, there was a lot of push back from browser vendors. Do you have any insight into why this problem was more visible to, more impactful for, developers? 
 
JG: Well, first off, not everybody was pushing back. Bruce Lawson [who works at Oprea] gets credit for coming up with the <picture> syntax. Other people were resistant, because they may not have seen the responsive design tidal wave that was coming.  So you have people who aren’t really thinking about mobile yet, or who were used to sites that were using device detection to route visitors to different, device-specific pages – and they couldn’t see how important this problem was, they didn’t see it as an issue.
 
Plus there are some tradeoffs that happen in responsive design that we still haven’t quite resolved.  By not supplying the height and width of images, we’re affecting browser’s ability to lay out pages quickly, and there’s a jumpy effect that happens when images come in. Browser makers were more attuned to those issues. And they had just finished doing a ton of performance optimization work to try to load images as quickly as possible, and, at first, it seemed like there was no way to do responsive images without breaking those optimizations.
 
So it wasn’t like browser vendors were saying, “we don’t want you in the clubhouse.” They really just had a different worldview. But the people who were building sites had come to realize that responsive design was inevitable and really important; that device diversity had gotten to the point where there was no point in making distinctions between a mobile site or a tablet site because some mobile devices were large and some tablets were small. But browser vendors weren’t there yet.
 
EP: Speaking for myself, and coming at it from the developer side, I know that I didn’t understand anything about low-level browser performance, and had no concept of how layout really worked at a deep level.  I was demanding tools and not understanding their potential impact – I couldn’t see the bigger picture. And it took a while to have that conversation – for the people who make web sites and the people who make web browsers to get into each other’s heads.
 
JG: In May 2012 I wrote about the preparser, two years after I started looking at responsive image issues. It took me that long to get to the point where I fundamentally understood that what we were talking about was a conflict between the browser, which wants to start downloading assets as soon as possible – before it starts laying out the page – and developers working on responsive designs, who want to delay and only start downloading images after layout. Because if we wait until after layout, we can download an image that fits. That fundamental conflict is what took so long to solve, from 2010 when we started identifying these problems, until 2014 when solutions started rolling out in browsers.
 
EP: And full support for the responsive images spec only rolled out in the last couple of browsers – mobile and desktop Safari – this past February. That felt amazing, like — ok, finally, it took four or five years but — we did it! This is complete.
 
JG: You know I had to write a bio for an article that just came out in A List Apart, and I ended it with: “He never wants to talk about responsive images again—”
 
EP: [Laughs] Sorry.
 
JG:“—he never wants to talk about responsive images again, but he has ideas for two more blog posts.”
 
[More laughs]
 
JG: Which is true. I really do have ideas for two more.
 
Thus concludes part 1 of my three-part interview with Jason Grigsby. Stay tuned for part 2, where Jason and I dive into today’s solutions and the prickly problem of Responsive Image Breakpoints.
 

Recent Blog Posts

Why the Future of E-commerce Is Live

In a previous post, I discussed how “going live” is gaining popularity across industries and verticals. What began as a way for gamers to jam together has evolved into a medium for broader entertainment and business purposes. To continue the conversation, this post unpacks the current trends of shoppable live streams to shine a light on how brands are leveraging “lives” to connect with shoppers in new ways.

Read more
An Overview of Live-Streaming Video Trends

“Let’s go live.” For decades, that’s what newscasters say as they cut to real-time footage of a colleague reporting in the field. The live-video feed adds visual interest and perspective to a story beyond what can be communicated by someone sitting behind the news desk. In the same way, live-streaming video nowadays adds context to other consumer environments. From gaming and events to shopping and social media, “going live” enhances everyday experiences, and it’s something anyone can do with relative ease.

Read more
Readying Live Streams for Video on Demand

When planning a live broadcast or stream, companies often overlook the redistribution phase, but live-stream videos are useful well beyond their initial streaming. Why? Because not everyone watches the first run. For a wider audience, it makes sense to repost live content on your website under an “events” tab, on YouTube, and other social sites for video on demand (VOD). However, preparing footage for reposting can be a lot of work.

Read more
Optimize Visual Media for a Fast and Captivating Digital Experience

Did you know that humans process imagery 60,000 times faster than text? In fact, 90% of the information our brains process is visual, which makes it seem a no-brainer—pun intended—that brands are connecting with consumers through visual content online. However, adding media assets like images and videos to websites comes with a tradeoff: the more media, the heavier the site, which results in a noticeable slowdown in page loads and a reduction in content quality.

Read more