Cloudinary Blog

Generate Waveform Images from Audio with Cloudinary

Generate Waveform Images from Audio with Cloudinary

This is a reposting of an article written by David Walsh. Check out his blog HERE!
I've been working a lot with visualizations lately, which is a far cry from your normal webpage element interaction coding; you need advanced geometry knowledge, render and performance knowledge, and much more. It's been a great learning experience but it can be challenging and isn't always an interest of all web developers. That's why we use apps and services specializing in complex tasks like Cloudinary: we need it done quickly and by a tool written by an expert.

While my previous experiments have been with images (Image Optimization, Remove Photo Backgrounds, and Automatic Image Tagging), Cloudinary also has the ability to manipulate video and audio files, as well as optimize delivery. This next experiment will mix imagery and media: we'll generate waveform images from an audio file!

Step 1: Upload the File

The first step is uploading the media file to Cloudinary, which you can automate with code or manually do so within the Cloudinary control panel. Let's presume the file is up on Cloudinary.

Step 2: Generate Image

You can use any number of languages to interact with Cloudinary's API but for the sake of this experiment we'll use Node.js and JavaScript. And the JavaScript required to generate and retrieve the basic waveform image? Much less than you think:

var result = cloudinary.image("Lights_qh6vve.png", {
	height: 200,
	width: 500,
	flags: "waveform",
	resource_type: "video"
});

So what exactly happens with the code above? Let's go through it:

  • The first argument, Lights_qh6vve.png, is the name of the uploaded MP3 file, replacing .mp3 with .png
  • The second argument provides the desired image settings, customizing height and width of generated image...
  • ...while flags: waveform and resource_type: video let Cloudinary know you want to generate the waveform image

The result is an img tag:

<img src='https://res.cloudinary.com/david-wash-blog/video/upload/fl_waveform,h_200,w_500/Lights_qh6vve.png' height='200' width='500'/>

..which looks like:

Customizing the Image

Cloudinary provides flexibility in image generation so let's create a more customized waveform image. Let's play with the colors:

var result = cloudinary.image("Lights_qh6vve.png", {
	height: 200,
	width: 500,
	flags: "waveform",
	resource_type: "video",
	background: '#acd7e5',
	color: '#ffda7f'
});

These colors generate a waveform image that looks like this:

Next we can use offset properties to get just a snippet of the waveform image:

var result = cloudinary.image("Lights_qh6vve.png", {
	height: 200,
	width: 500,
	flags: "waveform",
	resource_type: "video",
	background: '#acd7e5',
	color: '#ffda7f',
	start_offset: 1, // in seconds
	end_offset: 240
});

Which gives us this sharp image:

This experimentation was a lot of fun, and proves waveform image creation is just another amazing function provided by Cloudinary. Cloudinary is (an awesome) one-stop shop for uploading manipulating and delivering images and video. If you need to manipulate image or simply think you may need to do so in the future, give Cloudinary a good look -- they will do more than you think!

 

David Walsh David Walsh is Senior Software Engineer at Mozilla, having worked extensively on the Mozilla Developer Network, Firefox OS TV, WebVR, internal tooling, and several other Mozilla efforts. He shares his knowledge on his blog at http://davidwalsh.name. You can also find him at @davidwalshblog on Twitter.

Recent Blog Posts

Hipcamp Optimizes Images and Improves Page Load Times With Cloudinary

When creating a website that allows campers to discover great destinations, Hipcamp put a strong emphasis on featuring high-quality images that showcased the list of beautiful locations, regardless of whether users accessed the site on a desktop, tablet, or phone. Since 2015, Hipcamp has relied on Cloudinary’s image management solution to automate cropping and image optimization, enabling instant public delivery of photos, automatic tagging based on content recognition, and faster loading of webpages. In addition, Hipcamp was able to maintain the high standards it holds for the look and feel of its website.

Read more
New Image File Format: FUIF: Why Do We Need a New Image Format

In my last post, I introduced FUIF, a new, free, and universal image format I’ve created. In this post and other follow-up pieces, I will explain the why, what, and how of FUIF.

Even though JPEG is still the most widely-used image file format on the web, it has limitations, especially the subset of the format that has been implemented in browsers and that has, therefore, become the de facto standard. Because JPEG has a relatively verbose header, it cannot be used (at least not as is) for low-quality image placeholders (LQIP), for which you need a budget of a few hundred bytes. JPEG cannot encode alpha channels (transparency); it is restricted to 8 bits per channel; and its entropy coding is no longer state of the art. Also, JPEG is not fully “responsive by design.” There is no easy way to find a file’s truncation offsets and it is limited to a 1:8 downscale (the DC coefficients). If you want to use the same file for an 8K UHD display (7,680 pixels wide) and for a smart watch (320 pixels wide), 1:8 is not enough. And finally, JPEG does not work well with nonphotographic images and cannot do fully lossless compression.

Read more
 New Image File Format: FUIF:Lossy, Lossless, and Free

I've been working to create a new image format, which I'm calling FUIF, or Free Universal Image Format. That’s a rather pretentious name, I know. But I couldn’t call it the Free Lossy Image Format (FLIF) because that acronym is not available any more (see below) and FUIF can do lossless, too, so it wouldn’t be accurate either.

Read more
Optimizing Video Streaming and Delivery: Q&A with Doug Sillars

Doug Sillars, a digital nomad and a freelance mobile-performance expert, answers questions about video streaming and delivery, website optimization, and more.

Doug Sillars, a freelance mobile-performance expert and developer advocate, is a Google Developer Expert and the author of O’Reilly’s High Performance Android Apps. Given his extensive travels across the globe—from the UK to Siberia—with his wife, kids, and 11-year-old dog, Max, he has been referred to as a “digital nomad.” So far in 2018, Doug has spoken at more than 75 meetups and conferences!

Read more