Cloudinary Blog

Making Media Accessible: How to Automatically Generate alt Text for Images

How to Automatically Generate alt Text for Images

Images and videos are critical for ensuring user engagement on the web. For instance, on a retail website, images of a product from different angles or a 360-degree video of the product can lead to higher conversion rates. For a news website, users are more likely to read articles with visual media accompanying the content. It has been reported that posts that include images produce a 650-percent higher user-engagement rate than text-only posts.

Communicating intent to users through contextual images and videos is important. Most websites focus on identifying the context for users who can then visually and auditorily interpret the media. But what about handicapped users? It is equally important to communicate context to those whose disabilities prevent them from interpreting media on websites and mobile apps. The American Disabilities Act (ADA) makes it illegal for any government agency or business in the United States to offer to the public goods and services that are inaccessible to people with disabilities. Entities that offer services and products on the web must comply with the technical requirements in the Web Content Accessibility Guidelines (WCAG) 2.0 Level AA for digital accessibility.

Webinar
How to Optimize for Page Load Speed

The All-Important alt Text

One of the most common pitfalls in enabling accessible content is the lack of alt text in the HTML attribute in the source. Text embedded under alt specifies alternative verbiage that is rendered in place of the related image if it cannot be rendered. More importantly, screen-reader software uses alt text to enable someone who’s listening to the content of a web page, such as a blind person, to understand and interact with the content. According to a survey conducted by WebAIM.org, 89.2 percent of the respondents use a screen reader due to a disability.

This post demonstrates how you can leverage Cloudinary add-ons and third-party APIs to automate the process of generating alt text for media assets as part of your media-management pipeline. Through computer vision and machine-learning algorithms, those add-ons and APIs automatically generate keywords and phrases for alt text to scale—a tremendous time saver if you have thousands of assets.

Here are the Cloudinary add-ons for media recognition and categorization that can automate the generation of alt text:

Alternatively, several third-party services offer APIs for media recognition and categorization. Some examples are CloudSight, Clarifai.ai, and IBM Watson. Also, by using webhooks on Cloudinary, you can build, train, and deploy your own machine-learning models on most cloud providers that can be integrated into the pipeline.

Example

The example below shows you how to automatically generate alt text with Cloudinary’s Amazon Rekognition Auto Tagging add-on and CloudSight.

First, subscribe to the add-on and, if applicable, the service of your choice. Afterwards, tagging and maintaining context to generate alt text for images is extremely simple as part of the upload API call to Cloudinary, as shown below:

Ruby:

Copy to clipboard
Cloudinary::Uploader.upload("ice_skating.jpg", 
  :categorization => "aws_rek_tagging", :auto_tagging => 0.95, :notification_url => "https://mysite/my_notification_endpoint")

PHP:

Copy to clipboard
\Cloudinary\Uploader::upload("ice_skating.jpg", 
  array("categorization" => "aws_rek_tagging", "auto_tagging" => 0.95, "auto_tagging" => “https://mysite/my_notification_endpoint“));

Python:

Copy to clipboard
cloudinary.uploader.upload("ice_skating.jpg",
  categorization = "aws_rek_tagging", auto_tagging = 0.95,
  notification_url = "https://mysite/my_notification_endpoint")

Node.js:

Copy to clipboard
cloudinary.uploader.upload("ice_skating.jpg", 
  function(result) { console.log(result); }, 
  { categorization: "aws_rek_tagging", auto_tagging: 0.95,categorization: "https://mysite/my_notification_endpoint"});

Java:

Copy to clipboard
cloudinary.uploader().upload("ice_skating.jpg", ObjectUtils.asMap(
  "categorization", "aws_rek_tagging", "auto_tagging", "0.95", "notification_url", "https://mysite/my_notification_endpoint"));

The diagram below illustrates a sample workflow for Amazon Rekognition and CloudSight. The green arrows depict the processes that occur synchronously with respect to the upload API call; the orange arrows, those that occur asynchronously.

Sample workflow

The process proceeds as follows, step by step:

  1. You upload images to Cloudinary with the upload API.
  2. Cloudinary invokes the webhook specified in the notification_url parameter.
  3. Webhook invokes the CloudSight API to request a natural-language description of the uploaded images. Refer to the CloudSight API documentation for details.
  4. After receiving the description, the webhook updates the context of the images with the explicit API according to the tags from Amazon Rekognition and CloudSight’s natural-language description.
  5. When publishing the images to your website, you obtain their context through the Admin API.

See the interactive demo below for a few examples of images with pre-generated alt text. You can upload an image of your own and have Amazon Rekognition and CloudSight generate its alt text. Start by clicking Upload under Image. The code-level implementation of the demo is different from the workflow described above.

CodePen Demo here

See the Pen yxBGqV by Harish Jakkal (@harishcloudinary) on CodePen.

Conclusion

Generating meaningful alt text for a large volume of images to meet WCAG’s legal requirements for digital accessibility can be a daunting task. Not so with Cloudinary, as shown in this post. You can now take a step in the right direction by automating the production of alt text with Cloudinary add-ons or a third-party service of your choice.

That way, you can make media more accessible, simultaneously stimulating and enriching everyone’s browsing experience. What a worthwhile undertaking!

About Cloudinary

Cloudinary provides easy-to-use, cloud-based media management solutions for the world’s top brands. With offices in the US, UK and Israel, Cloudinary has quickly become the de facto solution used by developers and marketers at major companies around the world to streamline rich media management and deliver optimal end-user experiences.

For more information, visit www.cloudinary.com or follow us on Twitter.

Recent Blog Posts

New Learning Pathways From the Cloudinary Academy

In December 2019, Cloudinary launched its customer education platform, the Cloudinary Academy, replete with courses taught by the company’s experts on developer-oriented products and digital asset management (DAM) solution. The courses comprise interactive lessons and hands-on assignments, a proven way of familiarizing the audience with the course material and illustrating it with live examples.

Read more
Maya Shavin: How I Built My Website

Besides working as a senior front-end developer at Cloudinary, I'm also a content creator, a blogger, and an open-source developer. Follow me at @mayashavin and on mayashavin.com.

In the beginning, my website, mayashavin.com, was mainly for showcasing the status of my development projects and keeping me organized with my speaking schedule. Initially, I built it with Vue.js, later on switching to Nuxt.js (aka Nuxt) for a higher SEO score, and deployed it with Netlify. After some time, I added a blog section with Netlify CMS as the content management system (CMS). Everything was fine until I added more content and features, which led to a significant decline in the site’s performance. Also, the site design needed a modern look. So, I gave the site a makeover.

Read more
Automation Frees Up PetRescue’s Staff to Help Pets Find Their Forever Homes

As we spend more time at home, many of us are adopting pets for the joy, companionship and a surprising range of health benefits. In Australia, where our nonprofit customer PetRescue is located, there’s a shortage of pets to adopt. Last August, the Guardian reported that dog shelters in Australia emptied and adoption fees for puppies were running as high as $AUS1800.

Read more
Cloudinary and Contentful Make Modern Content Management Easier

I am pleased to share that Cloudinary and Contentful have joined forces to further streamline the creation, processing, and delivery of online content through Cloudinary’s digital asset management (DAM) solution and advanced transformation and delivery capabilities for images and video. What’s more, the partnership delivers a headless approach to DAM. By leveraging APIs for media management tasks, marketers and developers alike benefit from an integrated stack of optimized assets for optimization and automation. As a result, page loads are fast and beautiful, and at scale—with less overhead and effort.

Read more
Introducing Cloudinary's Nuxt Module

Since its initial release in October 2016 by the Chopin brothers as a server-side framework that runs on top of Vue.js, Nuxt (aka Nuxt.js) has gained prominence in both intuitiveness and performance. The framework offers numerous built-in features based on a modular architecture, bringing ease and simplicity to web development. Not surprisingly, Nuxt.js has seen remarkable growth in adoption by the developer community along with accolades galore. At this writing, Nuxt has earned over 30K stars on GitHub and 96 active modules with over a million downloads per month. And the upward trend is ongoing.

Read more