Cloudinary Blog

Automatic image categorization and tagging with Imagga

Automatic image tagging and categorization

If you have an application that allows users to upload their own photos, it can be very useful to be able to organize these photos according to their content. This will allow you to categorize the content for displaying to all your users and make your image library searchable. Furthermore, you can also learn more about your users according to the content they upload and find different trends of what people care about. Other added benefits can also include the ability to display matching content to your users according to their interests or even match them with other users that share similar interests.

Identifying images and their content is a process that would take huge amounts of time and resources if performed manually for many images. On the other hand, unlike text with words, images are data files with no meaning for simple software based filtering and require a deeper analysis of the actual meaning of the pixel colors in the image in order to automate the categorization and tagging process.

Imagga is a prominent company that develops and offers technologies, services, and online tools for visual image recognition, with technology that includes state-of-the-art machine learning approaches that allow it to be trained in the identification of various visual objects and concepts. In this post we would like to introduce you to a new add-on for Imagga's automatic image tagging capabilities, fully integrated into Cloudinary's image manipulation and management pipeline.

The Imagga Auto Tagging add-on

The Imagga add-on can automatically tell you what's in a photo by returning a list of detected tags and the confidence score of each tag.

The Imagga add-on is very simple to use: just set the categorization parameter of Cloudinary's image upload API to imagga_tagging while uploading an image and the response will include a list of suitable tags for the image.

For example, uploading the following picture of a koala to Cloudinary and requesting Imagga categorization:

Koala uploaded image for auto tagging

Ruby:
Cloudinary::Uploader.upload("koala.jpg",   
  :categorization => "imagga_tagging")
PHP:
\Cloudinary\Uploader::upload("koala.jpg",   
  array("categorization" => "imagga_tagging"));
Python:
cloudinary.uploader.upload("koala.jpg",  
  categorization = "imagga_tagging")
Node.js:
cloudinary.uploader.upload("koala.jpg",   
  function(result) { console.log(result); },   
  { categorization: "imagga_tagging" });
Java:
cloudinary.uploader().upload("koala.jpg", ObjectUtils.asMap(  
  "categorization", "imagga_tagging"));

The response will include the automatic categorization identified by the Imagga add-on. As can be seen in the response snippet below, various categories were automatically detected in the uploaded photo. The confidence score is a numerical value that represents the confidence level of the detected category, where 1.0 means 100% confidence. So Imagga is 100% sure that the picture contains a koala and only 8.5% sure that the picture contains a baboon.


"info"=>{
  "categorization"=>{
    "imagga_tagging"=>{
      "status"=>"complete", 
      "data"=>[
        {"tag"=>"koala", "confidence"=>1.0}, 
        {"tag"=>"mammal", "confidence"=>0.3151}, 
        {"tag"=>"monkey", "confidence"=>0.0882}
        {"tag"=>"baboon", "confidence"=>0.0853}

You can also get a list of tags from an already uploaded image by using the update method in Cloudinary's Admin API together with the image's public ID. For example, requesting Imagga categorization for the already uploaded image with a public ID of landscape:

Landscape photo for auto image tagging

Ruby:
Cloudinary::Api.update("landscape",   
  :categorization => "imagga_tagging")
PHP:
\Cloudinary\Api::update("landscape",   
  array("categorization" => "imagga_tagging"));
Python:
cloudinary.api.update("landscape",  
  categorization = "imagga_tagging")
Node.js:
cloudinary.api.update("landscape",   
  function(result) { console.log(result); },   
  { categorization: "imagga_tagging"});
Java:
cloudinary.api().update("landscape", ObjectUtils.asMap(  
  "categorization", "imagga_tagging"));

As with the upload API, the response includes the automatic categorization identified by Imagga shown in the response snippet below, where, for example, "landscape" is identified with 54% confidence, "grass" is identified with 44% confidence, along with a variety of other categories that may be relevant depending on your categorization scheme.


"info"=>{
  "categorization"=>{
    "imagga_tagging"=>{
      "status"=>"complete", 
      "data"=>[
        {"tag"=>"landscape", "confidence"=>0.5475}, 
        {"tag"=>"grass", "confidence"=>0.4414}, 
        {"tag"=>"field", "confidence"=>0.4121}, 
        {"tag"=>"sky", "confidence"=>0.394}, 
        {"tag"=>"land", "confidence"=>0.3514}, 
        {"tag"=>"rural", "confidence"=>0.3498}, 
        {"tag"=>"grassland", "confidence"=>0.3464}, 
        {"tag"=>"meadow", "confidence"=>0.3403}

Automatic tagging

If you want to organize, browse and manage your images based on the categories identified by Imagga, you can also automatically assign Cloudinary's resource tags to uploaded images.

Add the auto_tagging API parameter and set it to a minimum confidence level threshold, where all detected categories with a confidence level above this value will be automatically assigned that category as a resource tag.

For example, to automatically add tags to the landscape image with all detected categories that have a score higher than 0.4.

Ruby:
Cloudinary::Uploader.upload("landscape.jpg", 
  :categorization => "imagga_tagging", :auto_tagging => 0.4)
PHP:
\Cloudinary\Uploader::upload("landscape.jpg", 
  array("categorization" => "imagga_tagging", "auto_tagging" => 0.4));
Python:
cloudinary.uploader.upload("landscape.jpg",
  categorization = "imagga_tagging", auto_tagging = 0.4)
Node.js:
cloudinary.uploader.upload("landscape.jpg", 
  function(result) { console.log(result); }, 
  { categorization: "imagga_tagging", auto_tagging: 0.4 });
Java:
cloudinary.uploader().upload("landscape.jpg", ObjectUtils.asMap(
  "categorization", "imagga_tagging", "auto_tagging", "0.4"));

The response of the upload API call above returns the detected categories as well as the assigned tags. In this case, only the 'landscape', 'grass' and 'field' categories have a high enough score to be used as tags.

{ 
  ...    
  "tags" => [ "landscape", "grass", "field" ]   
  ...  
}

You can also use the update method to apply Imagga auto tagging to already uploaded images, based on their public IDs, and then automatically tag them according to the detected categories. See the Imagga add-on documentation for more information.

Categorization of images made easy

Categorizing photos can be a very useful and powerful tool that you can utilize to understand and organize the images that are uploaded by your users. Using the Imagga add-on as part of Cloudinary's service is simple and streamlined: you just need to add a parameter as part of the Cloudinary image upload and manipulation process.

Imagga add-on plans

The Imagga Auto Tagging add-on is available now and all Cloudinary plans can try it out with the add-on’s free tier. If you don't have a Cloudinary account yet, you can easily sign up for a free account.

Recent Blog Posts

Build a WhatsApp Clone with Automatic Image Optimization

In the previous post, we showed how to upload images to a Cloudinary server. In this part, we will play with some of the features we see on the WhatsApp technology. After you or your users have uploaded image assets to Cloudinary, you can deliver them via dynamic URLs. You can include instructions in your dynamic URLs that tell Cloudinary to manipulate your assets using a set of transformation parameters. All image manipulations and image optimizations are performed automatically in the cloud and your transformed assets are automatically optimized before they are routed through a fast CDN to the end user for an optimal user experience. For example, you can resize and crop, add overlays, blur or pixelate faces, apply a variety of special effects and filters, and apply settings to optimize your images and to deliver them responsively.

Read more
With automatic video subtitles, silence speaks volumes

The last time you scrolled through the feed on your favorite social site, chances are that some videos caught your attention, and chances are, they were playing silently.

On the other hand, what was your reaction the last time you opened a web page and a video unexpectedly began playing with sound? If you are anything like me, the first thing you did was to quickly hunt for the fastest way to pause the video, mute the sound, or close the page entirely, especially if you were in a public place at the time.

Read more
Impressed by WhatsApp Tech? Build WhatsApp Clone with Media Upload

With more than one billion people using WhatsApp, the platform is becoming a go-to for reliable and secure instant messaging. Having so many users means that data transfer processes must be optimized and scalable across all platforms. WhatsApp is touted for its ability to achieve significant media quality preservation when traversing the network from sender to receiver, and this is no easy feat to achieve.

Read more
New Google-powered add-on for auto video categories and tags

Due to significant growth of the web and improvements in network bandwidth, video is now a major source of information and entertainment shared over the internet. As a developer or asset manager, making corporate videos available for viewing, not to mention user-uploaded videos, means you also need a way to categorize them according to their content and make your video library searchable. Most systems end up organizing their video by metadata like the filename, or with user-generated tags (e.g., youtube). This sort of indexing method is subjective, inconsistent, time-consuming, incomplete and superficial.

Read more

iOS Developer Camp: The Dog House

By Shantini Vyas
iOS Developer Camp: The Dog House

Confession: I’m kind of addicted to hackathons. Ever since graduating from Coding Dojo earlier this year, I’ve been on the hunt for new places to expand my skills and meet new people in the tech space. iOS Developer Camp’s 10th Anniversary event bowled me over. Initially, because of its length. 48 hours? Yeesh. I had no idea that those 48 hours would change my life. But let’s first get a little backstory on my favorite topic: dogs.

Read more