Cloudinary Blog

FUIF: New, Legacy Friendly Image File Format

FUIF: New, Legacy Friendly Image File Format

Part 1 of this series introduces the Free Lossy Image Format (FUIF), which I recently developed. Part 2 explains the why, what, and how of FUIF. This post, part 3 of the series, delves into how FUIF is legacy friendly, which I alluded to in Part 2.

A vast number of JPEG images exist out there, as well as devices that produce new JPEG images every day. If the only way to transcode a JPEG image to a new image file format is to decode the JPEG to RGB pixels (or to YCbCr) and encode that to the new format, then there’s a problem. The result is either a significant generation loss or new files that are actually larger than the originals.

Compatibility With JPEG

FUIF is compatible with JPEG in the sense that it supports the 8x8 discrete cosine transform (DCT) and the YCbCr color transform. That means FUIF can losslessly represent the actual information in JPEG images while still offering you most of its benefits, such as the following:

  • Responsive by design, not limited to a scale of 1:8
  • Minimal header overhead
  • Better compression results than those from JPEG, thanks to enhanced entropy coding

Comparing FUIF to Dropbox's Lepton, which also does lossless JPEG recompression, reveals that—

  • Lepton can usually obtain a somewhat better compression.
  • Lepton is faster.
  • Lepton is bit exact, not just image exact.

However, FUIF offers these benefits:

  • You can decode FUIF progressively. Lepton is “antiprogressive” because it encodes the AC coefficients before the DC coefficients.
  • You can enhance JPEGs, for example, by adding a lossless or lossy alpha channel to them.

Unlike JPEG XT, the FUIF bitstream is completely different from legacy JPEG and it is thus unreadable to JPEG decoders. Obviously, full backward compatibility is convenient but it has serious downsides. It implies that the compression density cannot be improved compared to legacy JPEG. It also means that none of the new features (like alpha or higher bit depth) are guaranteed to be rendered correctly since, at least initially, most decoders would simply ignore any extensions of the legacy JPEG bitstream. So, in my opinion, not being backward compatible is a better design choice.

While FUIF is not backward compatible with JPEG, it is “legacy friendly” in the sense that you can convert existing JPEG images losslessly and effectively to FUIF, potentially with added information like alpha or depth channels.

Effective Recompression of PNGs and GIFs

FUIF can also effectively recompress existing (animated or still) PNGs and GIFs. Those formats are lossless within their limitations so generation loss is not an issue. However, effectively recompressing them can be challenging for new image formats since the compression methods for PNGs and GIFs differ from those for most lossy formats. For example, for images or animations with few different colors, PNG or GIF can be smaller than JPEG even though they are lossless and JPEG is lossy.

You can effectively recompress PNGs and GIFs by means of two specific image transformations in FUIF:

  • Palette: Both PNG8 and GIF use a color palette of up to 256 colors, replacing the usual three-color channels with just a single palette-index channel. FUIF supports palettes of arbitrary sizes.
  • Matching: This second transformation removes repetitive pixel patterns in images or animations, encoding them just once and replacing the other occurrences with references. This mode results in compression gains similar to what you can obtain with the dictionary-style entropy encoding of PNG and GIF. Because this mode can also compensate for interframe motion, it also works well in animations.

You can encode FUIF-recompressed PNGs and GIFs in a progressive—and thus responsive—way, either lossless or lossy.

Resilience to Generation Loss

Being able to recompress JPEGs, PNGs, and GIFs effectively without introducing additional loss is one way in which FUIF preserves the legacy of the past. Another way in which FUIF is legacy friendly pertains to the future.

Ideally, lossy compression is applied only once on an image. You should edit only lossless images and, in the final step, apply lossy compression to make those image files smaller for distribution. In practice, however, lossless originals often get lost and all that’s available as source material for editing are lossy images.

Repeated lossy encoding leads to generation loss, also called the “photocopier effect.” Ironically, advanced, modern image file formats tend to suffer more from that problem than the older formats. That’s because the advanced formats use a stronger arsenal of compression tricks, which enable them to obtain better compression but which also frequently result in more serious cumulative errors.

Realistically, it’s hard to completely avoid generation loss in lossy compression. Nonetheless, you can minimize the loss with FUIF. Here is an enlightening demo that shows how FUIF and some other image file formats behave after 500 generations of recompression:

As you can see, FUIF is legacy friendly in two key ways: it can effectively recompress existing JPEG, PNG, and GIF images (the legacy of the past) without introducing additional loss, and it is resilient to generation loss (the legacy to the future). In the next post, I will talk about compression artifacts, including those for FUIF.

Note
Currently, FUIF is still a research prototype. It has not yet been released and the bitstream remains work in progress. After the FUIF release, we’ll link to the source-code repository here.

Recent Blog Posts

Auto-Generate Video Previews with Great Results Every Time

If you are an avid YouTube viewer or a Netflix binger, you might have noticed some fascinating changes in their latest preview thumbnails. They sure are attention grabbing, aren’t they?

Video previews are becoming more and more popular, not the least because they entice viewers, attract clickthroughs and, more often than not, gain a happy and loyal audience. Problem is, the task sounds like an art in itself. Just how can you go about creating appealing video previews without having to rely on movie professionals? Have you ever wished that you could quickly create video previews like those on YouTube and Netflix, but still rest assured that they'll include the good stuff?

Read more
Hipcamp Optimizes Images and Improves Page Load Times With Cloudinary

When creating a website that allows campers to discover great destinations, Hipcamp put a strong emphasis on featuring high-quality images that showcased the list of beautiful locations, regardless of whether users accessed the site on a desktop, tablet, or phone. Since 2015, Hipcamp has relied on Cloudinary’s image management solution to automate cropping and image optimization, enabling instant public delivery of photos, automatic tagging based on content recognition, and faster loading of webpages. In addition, Hipcamp was able to maintain the high standards it holds for the look and feel of its website.

Read more
New Image File Format: FUIF: Why Do We Need a New Image Format

In my last post, I introduced FUIF, a new, free, and universal image format I’ve created. In this post and other follow-up pieces, I will explain the why, what, and how of FUIF.

Even though JPEG is still the most widely-used image file format on the web, it has limitations, especially the subset of the format that has been implemented in browsers and that has, therefore, become the de facto standard. Because JPEG has a relatively verbose header, it cannot be used (at least not as is) for low-quality image placeholders (LQIP), for which you need a budget of a few hundred bytes. JPEG cannot encode alpha channels (transparency); it is restricted to 8 bits per channel; and its entropy coding is no longer state of the art. Also, JPEG is not fully “responsive by design.” There is no easy way to find a file’s truncation offsets and it is limited to a 1:8 downscale (the DC coefficients). If you want to use the same file for an 8K UHD display (7,680 pixels wide) and for a smart watch (320 pixels wide), 1:8 is not enough. And finally, JPEG does not work well with nonphotographic images and cannot do fully lossless compression.

Read more
 New Image File Format: FUIF:Lossy, Lossless, and Free

I've been working to create a new image format, which I'm calling FUIF, or Free Universal Image Format. That’s a rather pretentious name, I know. But I couldn’t call it the Free Lossy Image Format (FLIF) because that acronym is not available any more (see below) and FUIF can do lossless, too, so it wouldn’t be accurate either.

Read more