Cloudinary Blog

How to select the perfect image format

Learn how to choose the best image format for your use case
JPEGs, PNGs, and GIFs — oh my! Most web developers only learn which format to use through trial, error, and long experience, and not necessarily by fully understanding how these formats actually work.
 
In this article, we’ll take a high-level look at each formats’ compression algorithms, in order to understand how their differing strategies make them more or less appropriate for different kinds of images. Then, we’ll tour a few next-generation image formats (like WebP and JPEG-XR) and see how we can use smart servers to take advantage of these powerful (but not yet universally-supported) formats, today.
 
Let’s kick things off with a dear old friend…

JPEG

Allow me, for a moment, to spend some time appreciating the venerable, amazing, JPEG. The twenty-five-year-old bag-of-algorithms that is ISO 10918 aka the JPEG standard has stood the test of time.
 
So, how does it work? And what is it good for?
 
JPEG compression carves an image up into 8×8-pixel-blocks, and then does something a little bit crazy to them, with a whole lot of math. It converts each block’s pixels—lists of R, G, and B values, mapped to sequential points in space—into a list of coefficients, used in equations which describe the block in terms of waves of energy. In technical terms, the JPEG compression algorithm translates an image’s information from the spatial domain to the frequency domain.
 
Practically, this means that JPEG is very good at compressing continuous-tone images: images with a tremendous number of colors, and smooth transitions between them. In other words, JPEG assumes that your image is going to look more or less like a photograph.
Conversely, the JPEG format is terrible at compressing images with crisp edges or high-energy, noisy textures—it’ll put rings around sharp edges and blur out fine detail.
 
Good or bad for JPG
 
In order to compress these images well, we will try to employ different compression algorithms. Enter our next format:

GIF

In the early days of the web, if an image wasn’t a JPEG, it was a GIF.
 
The GIF format uses the LZW compression algorithm, which is far simpler than JPEG’s mathemagic. Essentially, the LZW algorithm scans over your image data and generates very short codes for the parts of it that repeat; LZW shortens repetition. Which means that the GIF format is good at compressing images that have large chunks of identical or repetitive data. Images that only have a few colors, in broad swaths, with sharp transitions between them can be stored efficiently and losslessly as GIFs.
 
And even though it’s a simple format, GIF sports a pair of fancy features: transparency and animation.
 
But… GIF is terrible at compressing things that have even a moderate number of colors; heck, the format has a baked-in, hard, 256-color limit. Converting an image with more than that number of colors into a GIF results in lossy posterization, which looks awful.
 
Good or bad for GIF
 
In short, GIF and JPEG have opposite and complementary strengths. They made a killer team in the web’s early days.
 
But unresolved patent questions surrounding the LZW algorithm inspired a few very smart individuals to take a second crack at designing a lossless image format for the web.

PNG

PNG excels with the same sorts of images that GIF does, and brings a few added benefits:
 
  • No 256 color limit
  • Alpha channel transparency (so a pixel can be partially-transparent, and not simply all-transparent or fully-opaque)
  • In all but a few edge cases, superior compression

 PNG Compression

This image from Wikipedia shows off PNG’s ability to compress a full-color image with semi-transparent pixels pretty dang well.

How does PNG beat GIF, when it comes to compression? By adding some layers to its compression stack…

First, the PNG algorithm tries to reduce the amount of data it will need to store by using the pixels that it already knows about to predict the ones that it doesn’t. The format features five different prediction strategies, but basically, the PNG assumes that pixels next to each other will be similar. If this assumption proves true, PNG saves data by only storing the difference between its prediction and the actual value; small numbers take up less space than large ones.

Second, PNG cuts out repetition by allowing the image to refer to previous, identical pixel sequences (instead of storing the same data twice) using an algorithm called LZ77. If you squint, PNG’s LZ77 and GIF’s LZW are achieving the same ends—cutting out repetition—but get there via means which are distinct enough to evade patent lawyers. Everybody wins!

And then, finally, having done all of that, PNG uses a process called “Huffman coding” to boil the remaining values down even further by generating the smallest-possible codes for the most common values (incidentally, the JPEG format uses Huffman coding as a last-step as well).

Combining all three of these (lossless) techniques provides huge benefits over GIF’s single strategy. And sophisticated tools can provide even greater compression ratios by altering the original image data lossily before it’s run through this gauntlet, in order to make it more compression-friendly.

Stepping back, all that you need to know is this: PNG will perform worse than JPEG when it comes to photographs, and better than GIF almost always. So, use it for images with crisp edges and broad swaths of solid color or precisely repeated patterns.

The next generation

As of this writing, those three formats—JPEG, GIF, and PNG—are the only image formats with universal support. Which is to say, they’re the only formats that developers can actually use. But new, cutting edge formats are already here—and they’re spectacular.

WEBP

WebP is an offshoot of Google’s WebM video format; the core of it’s compression strategy is predictive, which is to say, it takes the simple predictive strategy used by the PNG format to the next level. WebP uses one of up to sixteen different prediction strategies for every (variable-sized) block in the image, and can optionally either losslessly or lossily compress the residual difference between the predicted and actual values. The format’s relative complexity provides it with a lot of flexibility; it’s good for a wide variety of imagery (both graphic, if you choose its lossless settings, and photographic, if you go lossy), with (generally) better compression than either PNG or JPEG.

But, it’s Google’s format and it is only supported in Chrome at the moment.

JPEG-XR

Microsoft’s next-gen-format-of-choice, JPEG-XR layers a bunch of new techniques on top of the basic mechanics of JPEG compression, enabling:

  • Lossless compression
  • More efficient lossy compression
  • Alpha-channel semi-transparency

Like WebP, JPEG-XR is a lot more complex, performant, and less-well-supported than its predecessors. Right now, the format is only supported in Internet Explorer and Edge.

How to use the formats of tomorrow, today

Is there any way for us to use these next-gen formats, right now? There is!
 
New markup allows developers to supply the same image in multiple formats, and lets the browser decide which one to load out of the bunch. Unfortunately, this markup can get a little complex:
 
<picture>
<source type="image/webp" srcset="sunset.wepb" />
<source type="image/vnd.ms-photo" srcset="sunset.jxr" />
<img src="sunset.jpg" alt="A colorful sunset" />
</picture>
 
Fortunately, there’s another way forward. Front-end engineers can shift this complexity to the back-end, employing smart servers that can send different users different resources from the same URL.
 
Using a service like Cloudinary, developers can deploy dynamic, adaptively-compressed images by adding a few simple characters to their URLs. Stick f_auto onto a Cloudinary URL, and you get adaptability without adding any complexity. The picture markup above boils back down to:
 
<img src="https://res.cloudinary.com/eric-cloudinary/
image/upload/f_auto/sunset" alt="A colorful sunset" />
 
How does this work? Turns out, clients tell servers which formats they support when they request image resources from servers. A smart server, then, can send different clients different resources, based on the information contained within their requests.
 
And that’s not all that a smart server can do–when you tack on a q_auto too, Cloudinary will automatically figure out not only which format will work best for each client, but also which specific compression settings will work well for your particular image, saving you from having to remember anything at all about each formats’ complex internal workings. So! Stop reading, and sign up for a free account today.

Recent Blog Posts

Reimaging DAM--The Next-Gen DAM for Marketing & Dev

There are great digital asset management (DAM) products out there for uploading, storing, managing, organizing, and sharing digital assets. With Cloudinary's new end-to-end DAM solution, you can also upload and manage your assets efficiently, but the journey doesn't end there. It continues on to the development and delivery stages, so that your assets can be seamlessly manipulated, optimized, and delivered to create an engaging user experience that will in turn, increase conversion and loyalty.

Read more
Integrating Cloudinary with Forestry’s Media Library

At Forestry, we believe that there is a bright future for static HTML sites built with tools like Jekyll and Hugo. These tools can create sites that run well, and are easy to host and maintain, because they don’t require any server-side code.

Read more
Video Optimization With the HTML5 <video> Player

Lack of experience and compression knowhow can cause significant user-experience problems. For instance, on a major retail site, I recently ran into a 48 MB video-hero banner. Pulling out the video and encoding it as an H.264 MP4 reduces the size to 1.9 MB. So, despite the desire for more video content, developers have not yet caught up to best practices. How do we get the best of both worlds without creating a disaster like the one above?

Read more
Build a Facial Emotion Recognition Based Video Suggestion App

Developers are always looking for new and creative ways to deliver content that resonates with the way users feel. Often using the latest technical innovations the market has to offer such as Artificial Intelligence (AI) and Machine Learning (ML). What better way to demonstrate innovative uses of these technology in a consumer market than capturing expressions from your users and then serving content based on that expression!

Read more
Improve Customer Data Protection with GDPR Implementation

TL;DR

Yay! We've done it! Gold-Star for us! We've talked with all the people, made all the changes, paid all the lawyers and checked all the boxes. GDPR? ✅Done!

Not so fast. Of course, conforming to the GDPR regulations introduced in Europe is just the beginning. This is a process and a state of mind that must become part of our long-term cultural ethos.

Read more