Last month at the annual Worldwide Developers Conference, Apple announced its support of JPEG XL. As someone who helped create JPEG XL, I don’t think anyone was more pleased than me to hear this news.
For a standard that’s not even three years old, this was a major win. Or rather, it’s a win for the web community, for photographers and artists, as well as for those of us who created the codec.
Before diving into what the Apple announcement means for JPEG XL moving forward, let’s take a quick look at its origin.
JPEG XL development began in 2018, when the JPEG committee launched a call for proposals on next-generation image compression, to which seven proposals were submitted. Of the seven two stood out: Google’s PIK and Cloudinary’s FUIF proposal. The ingredients from both proposals were eventually combined and refined to design a new codec that was better than the sum of its parts.
By the end of 2020, the main technical work was done and the bitstream was frozen, i.e., no more changes would be made that would change the format from the decoder point of view.
In November 2020 Cloudinary added JXL support and early in 2021, I wrote the blog post Time for Next-Gen Codecs to Dethrone JPEG. In the piece I argued that modern codecs like JPEG XL can bring many benefits, and expressed the hope that they would be widely adopted.
In early April 2021, the Chrome browser added experimental support (behind a flag), even before the JPEG XL standard was officially published. (The final draft had been submitted to ISO, but it would still take until March 2022 before it was approved and published as the international standard ISO/IEC 18181.) Firefox followed suit quickly and added experimental support. Things were looking good.
Then, on Halloween 2022, Chrome developers suddenly announced that they would be removing JPEG XL support. This decision was quite unexpected and controversial. In my blog The Case for JPEG XL, I argued why this decision should be reversed. In December, Chrome developers provided test results that were used to justify the decision and invited feedback. I analyzed the results and pointed out several methodological flaws and oversights. So far, my feedback has been ignored.
Beyond browsers, adoption of JPEG XL continued, in particular in image authoring software like Serif Affinity, Adobe Camera Raw, GIMP, Krita, etc. Unfortunately, Chrome’s decision has slowed wider adoption in web browsers of JPEG XL.
On June 5, 2023, at Apple’s annual Worldwide Developers Conference (WWDC23), a slide was presented listing new features of the Safari browser, and “JPEG XL” was on the slide. Not only is Safari 17 adding JPEG XL support, the new versions of iOS, iPadOS, macOS, watchOS, and visionOS will support JPEG XL.
This was unexpectedly good news. None of the JPEG XL developers had anticipated that the first browser to support this format that was co-created by Google, would be Safari. This is, of course, great news for the adoption of JPEG XL and leads me to think the momentum is back!
Image compression is a crucial part of what we do at Cloudinary, and for that we need a deep understanding of when to use what codec — or rather, what encoder and settings. For that reason, we did a large experiment to create the Cloudinary Image Dataset (CID22), a big dataset of human-annotated compressed images. This helps us to better understand the impact compression has on perceptual quality.
Comparing image encoders is not an easy task. The distortions caused by different encoders are different and it is hard to predict the perceived image quality. Subjective testing is still the gold standard of image quality assessment and the testing methodology is still an active area of research, in which Cloudinary participates. (For example, in the AIC-3 project of the JPEG committee.)
Based on datasets of human-annotated images, objective metrics like SSIMULACRA 2 can be validated. While not perfect, metrics can be used at scale to test many more images and encoder configurations than what would be feasible in a subjective experiment.
In image compression, there is a trade-off to be made between three things:
- Compression. How much can the file size be reduced?
- Quality. To what extent is the image degraded in visual quality?
- Speed. How long does it take to encode an image?
The importance of these elements is relative to one another and depends on the specific use case. For the same visual quality, we can look at the performance of various encoders in terms of compression and speed. We use the SSIMULACRA v2.1 metric as an indicator of visual quality on the horizontal axis. On the vertical axis, we have the compression performance (as a percentage saving compared to unoptimized 4:4:4 JPEG) on the main plot on the left hand side. On the plot on the right hand side we have encoding speed expressed in megapixels per second (when using a single cpu core). The plot shows JPEG XL (red), AVIF/libaom (yellow), WebP (green) and mozjpeg (white), each at their default speed setting. The dotted lines use 4:2:0 chroma subsampling (yuv420) while the full lines use 4:4:4 (yuv444). The range goes from a very low quality (around q30 in libjpeg terms) to a very high quality (around q95 in libjpeg terms).
The plot shows aggregated data corresponding to about 300 different source images. Some conclusions:
- WebP does bring a significant gain compared to unoptimized JPEG: a saving of 25 to 35% at the lower end of the quality spectrum, though at the higher end the savings diminish, mostly because WebP’s forced limited-range yuv420 does become a limiting factor (at q90+, it struggles to even reach the quality of JPEG, which can do full-range yuv444).
- Compared to the optimized mozjpeg encoder though, which also brings significant gains compared to unoptimized JPEG, the additional gains WebP brings are not as large: only about 3 to 5% additional savings.
- AVIF does bring a significant additional saving over WebP of about 10 to 15%. Moreover, since AVIF does not have forced yuv420, it can still bring significant savings over unoptimized JPEG even in the high-quality range, unlike WebP. However, these savings do come at a cost in speed: AVIF encoding at default speed is an order of magnitude slower than WebP or mozjpeg.
- JPEG XL brings an additional saving over AVIF of about 5 to 10%. The additional saving is larger in the high-quality range than in the lower-quality range. Moreover, it does this at a more reasonable speed.
Because Cloudinary was involved in the development of JPEG XL, we were the first to provide support. To convert an image to JXL with Cloudinary, simply add f_jxl to the url or change its extension to .jxl. Especially if you have significant traffic from visitors using iOS or Safari, it will likely be worthwhile to serve JPEG XL images (as soon as the new iOS and Safari have landed), even if you still need AVIF, WebP, or JPEG fallbacks for other visitors.
While in terms of average compression performance, JPEG XL is currently the best codec available, there is a lot of image-dependent variation. For some images, JPEG XL is a lot better than the other codecs, while for other images, AVIF is better than JPEG XL. For this reason, we are currently working on an AI-powered new version of our f_auto,q_auto feature, which can automatically select the optimal format to use on an image-by-image basis.
Have any questions or want to discuss the topic of this blog in greater detail? The Cloudinary Community is a great place to share your questions, ideas, and suggestions. You can also ping me on the Cloudinary Community Discord server here: Jon: @_wb_
