Google Auto Tagging

Cloudinary is a cloud-based service that provides an end-to-end image management solution including uploads, storage, manipulations, optimizations and delivery. Cloudinary offers a very rich set of image manipulation and analysis capabilities and allows you to extract the semantic data from uploaded images: photo metadata (Exif & IPTC) including location and camera details, coordinates of automatically detected faces, color histogram and predominant colors. In addition, Cloudinary allows you to assign multiple tags to images for listing and managing your media library.

Google Cloud enables developers to understand the content of an image by utilizing powerful machine learning models and quickly classify images into thousands of categories. Cloudinary provides an add-on for Google's automatic image tagging capabilities, fully integrated into Cloudinary's image management and manipulation pipeline. Google analyses image data and automatically identifies categories and suggests tags, a process that would take huge amounts of time and resources if performed manually. The analyses also improves over time as new concepts are introduced and accuracy is improved.

With the Google auto tagging add-on, you can extend Cloudinary's powerful semantic data extraction and image tagging features. When using the Google auto tagging add-on, your images are automatically tagged according to the categories detected in each image.

Tip
You can optionally request the category and tag results in another language or in multiple languages by registering for the Google Translation Add-on in addition to the tagging add-on.

Image recognition and categorization

Take a look at the following photo of children ice skating:

Ruby:
cl_image_tag("ice_skating.jpg")
PHP:
cl_image_tag("ice_skating.jpg")
Python:
CloudinaryImage("ice_skating.jpg").image()
Node.js:
cloudinary.image("ice_skating.jpg")
Java:
cloudinary.url().imageTag("ice_skating.jpg");
JS:
cloudinary.imageTag('ice_skating.jpg').toHtml();
jQuery:
$.cloudinary.image("ice_skating.jpg")
React:
<Image publicId="ice_skating.jpg" >

</Image>
Angular:
<cl-image public-id="ice_skating.jpg" >

</cl-image>
.Net:
cloudinary.Api.UrlImgUp.BuildImageTag("ice_skating.jpg")
Android:
MediaManager.get().url().generate("ice_skating.jpg");
iOS:
imageView.cldSetImage(cloudinary.createUrl().generate("ice_skating.jpg")!, cloudinary: cloudinary)
ice_skating.jpg

By setting the categorization parameter to google_tagging when calling Cloudinary's upload or update methods, Google is used to automatically classify the scenes of the uploaded or specified existing image. For example:

Ruby:
Cloudinary::Uploader.upload("ice_skating.jpg", 
  :categorization => "google_tagging")
PHP:
\Cloudinary\Uploader::upload("ice_skating.jpg", 
  array("categorization" => "google_tagging"));
Python:
cloudinary.uploader.upload("ice_skating.jpg",
  categorization = "google_tagging")
Node.js:
cloudinary.v2.uploader.upload("ice_skating.jpg", 
  { categorization: "google_tagging" }),
  function(error, result) {console.log(result, error) });
Java:
cloudinary.uploader().upload("ice_skating.jpg", ObjectUtils.asMap(
  "categorization", "google_tagging"));
.Net:
var uploadParams = new ImageUploadParams() 
{
  File = new FileDescription(@"ice_skating.jpg"),
  Categorization = "google_tagging"
};
var uploadResult = cloudinary.Upload(uploadParams);

Tips

  • You can use upload presets to centrally define a set of upload options including add-on operations to apply, instead of specifying them in each upload call. You can define multiple upload presets, and apply different presets in different upload scenarios. You can create new upload presets in the Upload page of the Management Console settings or using the Admin API. From the Upload page of the console settings, you can also select a default upload preset to use for all API uploads and a default preset for all Media Library (UI) uploads.
  • You can run multiple categorization add-ons on the resource. The categorization parameter accepts a comma-separated list of all the Cloudinary categorization add-ons to run on the resource.

The upload API response includes the automatic categorization identified by the Google auto tagging add-on. As can be seen in the response snippet below, various categories were automatically detected in the uploaded photo. The confidence score is a numerical value that represents the confidence level of the detected category, where 1.0 means 100% confidence.

{
...
 "info":
  {"categorization":
    {"google_tagging":
      {"status": "complete",
       "data":
        [{"tag": "skating", "confidence": 0.9689},
         {"tag": "footwear", "confidence": 0.9587},
         {"tag": "ice skating", "confidence": 0.9513},
         {"tag": "ice rink", "confidence": 0.9469},
         {"tag": "ice skate", "confidence": 0.9271},
         {"tag": "winter", "confidence": 0.911},
         {"tag": "fun", "confidence": 0.8944},
         {"tag": "girl", "confidence": 0.8183},
         {"tag": "ice", "confidence": 0.8045},
         {"tag": "winter sport", "confidence": 0.7912},
         {"tag": "recreation", "confidence": 0.7568},
         {"tag": "child", "confidence": 0.7532},
         {"tag": "leisure", "confidence": 0.7076},
         {"tag": "play", "confidence": 0.6854},
         {"tag": "snow", "confidence": 0.6167},
         {"tag": "road", "confidence": 0.5823},
         {"tag": "sports", "confidence": 0.5223}]}}}

Adding resource tags to images

Automatically categorizing your images is a useful way to organize your Cloudinary media library, and by also providing the auto_tagging parameter to an upload or update call, images are automatically assigned resource tags based on the detected scene categories. The value of the auto_tagging parameter is the minimum confidence score of a detected category that should be automatically used as an assigned resource tag. Assigning these resource tags allows you to list and search images in your media library using Cloudinary's API and Web interface. The following code example automatically tags an uploaded image with all detected categories that have a confidence score higher than 0.6.

Ruby:
Cloudinary::Uploader.upload("ice_skating.jpg", 
  :categorization => "google_tagging", :auto_tagging => 0.6)
PHP:
\Cloudinary\Uploader::upload("ice_skating.jpg", 
  array("categorization" => "google_tagging", "auto_tagging" => 0.6));
Python:
cloudinary.uploader.upload("ice_skating.jpg",
  categorization = "google_tagging", auto_tagging = 0.6)
Node.js:
cloudinary.v2.uploader.upload("ice_skating.jpg", 
  { categorization: "google_tagging", 
    auto_tagging: 0.6 }),
  function(error, result) {console.log(result, error) });
Java:
cloudinary.uploader().upload("ice_skating.jpg", ObjectUtils.asMap(
  "categorization", "google_tagging", "auto_tagging", "0.6"));
.Net:
var uploadParams = new ImageUploadParams() 
{
  File = new FileDescription(@"ice_skating.jpg"),
  Categorization = "google_tagging",
  AutoTagging = 0.6
};
var uploadResult = cloudinary.Upload(uploadParams);

The response of the upload API call above returns the detected categories as well as the assigned tags for categories meeting the minimum confidence score of 0.6:

{ 
...    
"tags": [ "skating", "footwear", "ice skating", "ice rink", "ice skate", "winter", "fun", "girl", "ice", "winter sport", "recreation", "child", "leisure", "play", "snow" ]
...
}

You can also use the update method to apply Google auto tagging to already uploaded images, based on their public IDs, and then automatically tag them according to the detected categories. For example, the following image was uploaded to Cloudinary with the 'puppy' public ID.

Ruby:
cl_image_tag("puppy.jpg")
PHP:
cl_image_tag("puppy.jpg")
Python:
CloudinaryImage("puppy.jpg").image()
Node.js:
cloudinary.image("puppy.jpg")
Java:
cloudinary.url().imageTag("puppy.jpg");
JS:
cloudinary.imageTag('puppy.jpg').toHtml();
jQuery:
$.cloudinary.image("puppy.jpg")
React:
<Image publicId="puppy.jpg" >

</Image>
Angular:
<cl-image public-id="puppy.jpg" >

</cl-image>
.Net:
cloudinary.Api.UrlImgUp.BuildImageTag("puppy.jpg")
Android:
MediaManager.get().url().generate("puppy.jpg");
iOS:
imageView.cldSetImage(cloudinary.createUrl().generate("puppy.jpg")!, cloudinary: cloudinary)
sample.jpg

The following code sample uses Cloudinary's update method to apply Google's automatic image tagging and categorization to the puppy uploaded image, and then automatically assign resource tags based on the categories detected with over a 90% confidence level.

Ruby:
Cloudinary::Api.update("puppy", 
  :categorization => "google_tagging", 
  :auto_tagging => 0.9)
PHP:
\Cloudinary\Api::update("puppy", 
  array(
    "categorization" => "google_tagging", 
    "auto_tagging" => 0.9));
Python:
cloudinary.api.update("puppy",
  categorization = "google_tagging", 
  auto_tagging = 0.9)
Node.js:
cloudinary.v2.api.update("puppy", 
  { categorization: "google_tagging", 
    auto_tagging: 0.9 }),
  function(error, result) {console.log(result, error) });
Java:
cloudinary.api().update("puppy", ObjectUtils.asMap(
  "categorization", "google_tagging", 
  "auto_tagging", 0.9));
.Net:
var updateParams = new UpdateParams("puppy") 
{
  Categorization = "google_tagging",
  AutoTagging = 0.9
};
var updateResult = cloudinary.UpdateResource(updateParams);

The response of the update API call includes the detected categories, and automatically assigned tags. As you can see in the response snippet below, tags were only added for the scene categories with a confidence score of over 90%.

{
  ...
  "tags": [ "dog", "dog like mammal", "dog breed", "mammal" ]
  "info": {
    "categorization": {
      "google_tagging": {
        "status": "complete",
        "data": [
         {"tag": "dog", "confidence": 0.9611},
         {"tag": "dog like mammal", "confidence": 0.9508},
         {"tag": "dog breed", "confidence": 0.9396},
         {"tag": "mammal", "confidence": 0.923},
         {"tag": "dog breed group", "confidence": 0.8883},
         {"tag": "borador", "confidence": 0.8145},
         {"tag": "grass", "confidence": 0.7731},
         {"tag": "puppy", "confidence": 0.7715},
         ...
}

You can use the Admin API's resource method to return the details of a resource, including the the scene categories that you already extracted using the upload or update methods.

Ruby:
Cloudinary::Api.resource("puppy")
PHP:
\Cloudinary\Api::resource("puppy");
Python:
cloudinary.v2.api.resource("puppy")
Node.js:
cloudinary.api.resource("puppy", 
  function(result) { console.log(result); });
Java:
cloudinary.api().resource("puppy", ObjectUtils.emptyMap());
.Net:
var getResourceParams = new GetResourceParams("puppy");
var getResourceResult = cloudinary.GetResource(getResourceParams);