Cloudinary Blog
What to consider when developing media rich websites and apps
Websites have evolved greatly over the past few years. Once text-heavy websites have become more eye-catching with prominent images and video. But the addition of richer media isn’t the only change impacting websites. Consumer behavior also factors into this evolution, as web access has moved from the desktop realm to a variety of different devices – smartphones, tablets, laptops, smartwatches and TVs – with different dimensions and resolutions. And consumers want to be able to access web content anytime and from any location.
 
To ensure that website performance is optimized, bandwidth usage is minimized and users have a top-notch experience, we will need to address many challenges.

High resolution images and videos

If a picture is worth 1,000 words, it’s no surprise that website owners are using images and videos as powerful tools on their websites to engage visitors. High-resolution images look best, but these files are so big that they cause websites to load slowly and use excessive bandwidth. These problems frustrate visitors, and potentially decrease their incentive to  engage further on the website.
 
Loading image
 
Whether your users are visiting your site from a phone or a computer, it’s imperative that it loads quickly. Gabriel A. Mays, founder of Just Add Content, a website platform for businesses, told CIO magazine that developers should “Aim to keep website load time to [a few] seconds or less. Your biggest threat isn't a competitor, it's the back button. If your website loads too slowly, customers won't wait around. They'll go elsewhere.”
 
When addressing these issues:
  • Resize images/videos to match device resolution – One size doesn’t fit all, particularly with the increasing number of devices of different sizes being adopted by consumers.
  • Leverage modern image formats or video codecs – For images, consider using WebP, with automatic fallback to JPEG or other formats for browsers that don’t support newer formats. With video, consider the codecs, frame rate and bit-rate to save file size and bandwidth.
  • Adjust the quality level – There is a tradeoff between compression levels and visual quality to ensure a satisfactory user experience without excessive bandwidth use.
New types of images are introduced almost daily, requiring we stay on trend and learn how to best display them on our websites. For example, Apple introduced Live Photo, kind of a hybrid between a static image and video, combining a photo with other moments before and after it was taken, and displays them with movement and sound. What is the best way to support uploading, transforming and displaying these new forms of content? How might one ensure that these images are bandwidth and storage efficient and visually appealing, regardless of the device where it’s viewed? 

Greater use of video - upstream and downstream

It’s undeniable that video is becoming a primary component on the web – from the videos uploaded by website owners to attract visitors, to the videos being uploaded by users to share with the public. By 2019,  nearly a million minutes of video content will cross the network every second , according to the Cisco Visual Networking Index
 
User uploaded videos
 
As users upload greater quantities of videos, so will grow the task of making videos of various qualities, dimensions and aspect ratios fit into the graphic design of a website or mobile app.
 
But it’s not just the number of minutes of video being uploaded, it’s the resolution. Today’s devices are made to handle high resolution video, and as a result, 4K video is increasing in popularity. But the huge resolution translates into long upload and download times; need for increased storage space; and intensive client-side processing that is required to convert, resize and manipulate these videos.
 
This requires us to normalize and optimize 4K and high-res video specifically for web and mobile devices, and leverage responsive technology that will enable us to deliver the smallest file size while still maintaining the highest visual quality to match the user’s device, browser and network speed.

Responsive design

Probably the most debated issue these days is responsive design. Responsive design enables the same website to adapt to different resolutions, with various techniques, ensuring images and videos look and operate properly on the plethora of devices in use today, at various resolutions. 
 
Google, as well as standards organization W3C, Microsoft and Apple, are all trying to simplify responsive design with solutions built into web browsers. But these aren’t sufficient. For example, the Client Hints solution that’s being promoted by Google is only supported in Chrome, so additional work is required to ensure your images are displayed properly on other browsers.
 
Many devices require responsive design
 
Another option includes relying on the new HTML5 features - specifically the <picture> element and the 'srcset' attribute of the <img> element - to define the various image resolutions and art-directed cropped and manipulated image versions within the HTML code. 
 
There are two problems with this approach: 
  1. Not all browsers support the modern HTML5 features, so workarounds must be developed as a fallback mechanism for browsers that don’t. However Google has been supporting modern HTML5 elements in Chrome for awhile, and Microsoft and Safari are adding support for responsive design to the latest versions of their browsers.
  2. While the browsers automatically select the best matching images for each device and resolution, the browser doesn’t automatically create the images. This requires double (or triple or quadruple…) the work, pre-creating multiple different image versions, or alternatively using a dynamic image manipulation service.
Additionally, these solutions do not focus on finding the appropriate Responsive Breakpoints. When creating a responsive website, choosing the correct image resolutions and how many different image versions to include in your responsive website is called Responsive Breakpoints. 
While breakpoints can technically be any size, ideally they should be set at the optimal resolutions and sizes of images needed to best fit the various devices and screen sizes on which your website will be viewed.
For determining breakpoints, developers need a solution that helps them decide which image resolutions are needed, create multiple images, and integrate with their HTML code or leverage Javascript solutions. 
 
To solve this issue, Cloudinary last year launched a Responsive Image Breakpoints Generator tool, which efficiently and intelligently calculates the image breakpoints.
 
Recently, we also launched our "Auto-everything" solution, taking Cloudinary's cloud-based image management solution to the next level using automatic content-aware and context-aware image adaptation. Within this, we introduced two new transformation parameters, which pair the 'DPR' and 'Width' Client Hints with our existing image resizing and delivery infrastructure, in order to serve up simple, automatic responsive images.

Moving forward

The evolution of video and image formats, coupled with constant innovation in devices and displays, will continue to raise challenges, as developers seek to create a superb user experience while minimizing the impact on bandwidth, storage and website performance.
 
Effectively managing high resolution files, adeptly handling the growing amount of video both incorporated in designs and uploaded by users, and incorporating responsive design techniques as described above can help address some of today’s challenges, and establish a good foundation for future best practices.

Recent Blog Posts

Hipcamp Optimizes Images and Improves Page Load Times With Cloudinary

When creating a website that allows campers to discover great destinations, Hipcamp put a strong emphasis on featuring high-quality images that showcased the list of beautiful locations, regardless of whether users accessed the site on a desktop, tablet, or phone. Since 2015, Hipcamp has relied on Cloudinary’s image management solution to automate cropping and image optimization, enabling instant public delivery of photos, automatic tagging based on content recognition, and faster loading of webpages. In addition, Hipcamp was able to maintain the high standards it holds for the look and feel of its website.

Read more
New Image File Format: FUIF: Why Do We Need a New Image Format

In my last post, I introduced FUIF, a new, free, and universal image format I’ve created. In this post and other follow-up pieces, I will explain the why, what, and how of FUIF.

Even though JPEG is still the most widely-used image file format on the web, it has limitations, especially the subset of the format that has been implemented in browsers and that has, therefore, become the de facto standard. Because JPEG has a relatively verbose header, it cannot be used (at least not as is) for low-quality image placeholders (LQIP), for which you need a budget of a few hundred bytes. JPEG cannot encode alpha channels (transparency); it is restricted to 8 bits per channel; and its entropy coding is no longer state of the art. Also, JPEG is not fully “responsive by design.” There is no easy way to find a file’s truncation offsets and it is limited to a 1:8 downscale (the DC coefficients). If you want to use the same file for an 8K UHD display (7,680 pixels wide) and for a smart watch (320 pixels wide), 1:8 is not enough. And finally, JPEG does not work well with nonphotographic images and cannot do fully lossless compression.

Read more
 New Image File Format: FUIF:Lossy, Lossless, and Free

I've been working to create a new image format, which I'm calling FUIF, or Free Universal Image Format. That’s a rather pretentious name, I know. But I couldn’t call it the Free Lossy Image Format (FLIF) because that acronym is not available any more (see below) and FUIF can do lossless, too, so it wouldn’t be accurate either.

Read more
Optimizing Video Streaming and Delivery: Q&A with Doug Sillars

Doug Sillars, a digital nomad and a freelance mobile-performance expert, answers questions about video streaming and delivery, website optimization, and more.

Doug Sillars, a freelance mobile-performance expert and developer advocate, is a Google Developer Expert and the author of O’Reilly’s High Performance Android Apps. Given his extensive travels across the globe—from the UK to Siberia—with his wife, kids, and 11-year-old dog, Max, he has been referred to as a “digital nomad.” So far in 2018, Doug has spoken at more than 75 meetups and conferences!

Read more