Cloudinary Blog

Automatic image categorization and tagging with Imagga

Automatic image tagging and categorization

If you have an application that allows users to upload their own photos, it can be very useful to be able to organize these photos according to their content. This will allow you to categorize the content for displaying to all your users and make your image library searchable. Furthermore, you can also learn more about your users according to the content they upload and find different trends of what people care about. Other added benefits can also include the ability to display matching content to your users according to their interests or even match them with other users that share similar interests.

Identifying images and their content is a process that would take huge amounts of time and resources if performed manually for many images. On the other hand, unlike text with words, images are data files with no meaning for simple software based filtering and require a deeper analysis of the actual meaning of the pixel colors in the image in order to automate the categorization and tagging process.

Imagga is a prominent company that develops and offers technologies, services, and online tools for visual image recognition, with technology that includes state-of-the-art machine learning approaches that allow it to be trained in the identification of various visual objects and concepts. In this post we would like to introduce you to a new add-on for Imagga's automatic image tagging capabilities, fully integrated into Cloudinary's image management and manipulation pipeline.

The Imagga Auto Tagging add-on

The Imagga add-on can automatically tell you what's in a photo by returning a list of detected tags and the confidence score of each tag.

The Imagga add-on is very simple to use: just set the categorization parameter of Cloudinary's image upload API to imagga_tagging while uploading an image and the response will include a list of suitable tags for the image.

For example, uploading the following picture of a koala to Cloudinary and requesting Imagga categorization:

Koala uploaded image for auto tagging

Ruby:
Cloudinary::Uploader.upload("koala.jpg",   
  :categorization => "imagga_tagging")
PHP:
\Cloudinary\Uploader::upload("koala.jpg",   
  array("categorization" => "imagga_tagging"));
Python:
cloudinary.uploader.upload("koala.jpg",  
  categorization = "imagga_tagging")
Node.js:
cloudinary.uploader.upload("koala.jpg",   
  function(result) { console.log(result); },   
  { categorization: "imagga_tagging" });
Java:
cloudinary.uploader().upload("koala.jpg", ObjectUtils.asMap(  
  "categorization", "imagga_tagging"));

The response will include the automatic categorization identified by the Imagga add-on. As can be seen in the response snippet below, various categories were automatically detected in the uploaded photo. The confidence score is a numerical value that represents the confidence level of the detected category, where 1.0 means 100% confidence. So Imagga is 100% sure that the picture contains a koala and only 8.5% sure that the picture contains a baboon.


"info"=>{
  "categorization"=>{
    "imagga_tagging"=>{
      "status"=>"complete", 
      "data"=>[
        {"tag"=>"koala", "confidence"=>1.0}, 
        {"tag"=>"mammal", "confidence"=>0.3151}, 
        {"tag"=>"monkey", "confidence"=>0.0882}
        {"tag"=>"baboon", "confidence"=>0.0853}

You can also get a list of tags from an already uploaded image by using the update method in Cloudinary's Admin API together with the image's public ID. For example, requesting Imagga categorization for the already uploaded image with a public ID of landscape:

Landscape photo for auto image tagging

Ruby:
Cloudinary::Api.update("landscape",   
  :categorization => "imagga_tagging")
PHP:
\Cloudinary\Api::update("landscape",   
  array("categorization" => "imagga_tagging"));
Python:
cloudinary.api.update("landscape",  
  categorization = "imagga_tagging")
Node.js:
cloudinary.api.update("landscape",   
  function(result) { console.log(result); },   
  { categorization: "imagga_tagging"});
Java:
cloudinary.api().update("landscape", ObjectUtils.asMap(  
  "categorization", "imagga_tagging"));

As with the upload API, the response includes the automatic categorization identified by Imagga shown in the response snippet below, where, for example, "landscape" is identified with 54% confidence, "grass" is identified with 44% confidence, along with a variety of other categories that may be relevant depending on your categorization scheme.


"info"=>{
  "categorization"=>{
    "imagga_tagging"=>{
      "status"=>"complete", 
      "data"=>[
        {"tag"=>"landscape", "confidence"=>0.5475}, 
        {"tag"=>"grass", "confidence"=>0.4414}, 
        {"tag"=>"field", "confidence"=>0.4121}, 
        {"tag"=>"sky", "confidence"=>0.394}, 
        {"tag"=>"land", "confidence"=>0.3514}, 
        {"tag"=>"rural", "confidence"=>0.3498}, 
        {"tag"=>"grassland", "confidence"=>0.3464}, 
        {"tag"=>"meadow", "confidence"=>0.3403}

Automatic tagging

If you want to organize, browse and manage your images based on the categories identified by Imagga, you can also automatically assign Cloudinary's resource tags to uploaded images.

Add the auto_tagging API parameter and set it to a minimum confidence level threshold, where all detected categories with a confidence level above this value will be automatically assigned that category as a resource tag.

For example, to automatically add tags to the landscape image with all detected categories that have a score higher than 0.4.

Ruby:
Cloudinary::Uploader.upload("landscape.jpg", 
  :categorization => "imagga_tagging", :auto_tagging => 0.4)
PHP:
\Cloudinary\Uploader::upload("landscape.jpg", 
  array("categorization" => "imagga_tagging", "auto_tagging" => 0.4));
Python:
cloudinary.uploader.upload("landscape.jpg",
  categorization = "imagga_tagging", auto_tagging = 0.4)
Node.js:
cloudinary.uploader.upload("landscape.jpg", 
  function(result) { console.log(result); }, 
  { categorization: "imagga_tagging", auto_tagging: 0.4 });
Java:
cloudinary.uploader().upload("landscape.jpg", ObjectUtils.asMap(
  "categorization", "imagga_tagging", "auto_tagging", "0.4"));

The response of the upload API call above returns the detected categories as well as the assigned tags. In this case, only the 'landscape', 'grass' and 'field' categories have a high enough score to be used as tags.

{ 
  ...    
  "tags" => [ "landscape", "grass", "field" ]   
  ...  
}

See the Imagga add-on documentation for more information.

Categorization of images made easy

Categorizing photos can be a very useful and powerful tool that you can utilize to understand and organize the images that are uploaded by your users. Using the Imagga add-on as part of Cloudinary's service is simple and streamlined: you just need to add a parameter while uploading and manipulating images via Cloudinary.

Imagga add-on plans

The Imagga Auto Tagging add-on is available now and all Cloudinary plans can try it out with the add-on’s free tier. If you don't have a Cloudinary account yet, you can easily sign up for a free account.

Recent Blog Posts

Build the Back-End For Your Own Instagram-style App with Cloudinary

Github Repo

Managing media files (processing, storage and manipulation) is one of the biggest challenges we encounter as practical developers. These challenges include:

A great service called Cloudinary can help us overcome many of these challenges. Together with Cloudinary, let's work on solutions to these challenges and hopefully have a simpler mental model towards media management.

Read more

Build A Miniflix in 10 Minutes

By Prosper Otemuyiwa
Build A Miniflix in 10 Minutes

Developers are constantly faced with challenges of building complex products every single day. And there are constraints on the time needed to build out the features of these products.

Engineering and Product managers want to beat deadlines for projects daily. CEOs want to roll out new products as fast as possible. Entrepreneurs need their MVPs like yesterday. With this in mind, what should developers do?

Read more

Your Web Image is Unnecessarily Bloated

By Christian Nwamba
Your Web Image is Unnecessarily Bloated

As a developer, it seems inefficient to serve a 2000kb JPEG image when we could compress images to optimize the performance without degrading the visual quality.

We are not new to this kind of responsibility. But our productivity will end up being questioned if we do not deliver fast. In order to do so, the community has devised several patterns to help improve productivity. Let's review few of these patterns based on their categories:

Read more

Google For Nigeria: We saw it all…

By Christian Nwamba
Google For Nigeria: We saw it all…

Note from Cloudinary: Christian Nwamba, a frequent Cloudinary contributor, recently attended, and was a main speaker, at the Google Developer Group (GDG) Conference in Lagos, Nigeria. Christian led a session teaching more than 500 developers how to “Build Offline Apps for the Next Billion Users.” The stack he used included JS (Vue), Firebase, Service Workers and Cloudinary. Below is his account of the conference and his talk.

Read more
Viral Images: Securing Images and Video uploads to your systems

When was the last time you got paid $40,000 for a few days of work? That is what happened last year to Russian independent security researcher Andrey Leonov, who discovered that if you upload a specially constructed image file to Facebook, you can make Facebook's internal servers, nested deep within their firewalls, run arbitrary commands to expose sensitive internal files in a way that could easily lead to a data breach.

Read more