An ever-increasing amount of visual data is created and stored due to the rapid evolution of digital technology. Therefore, there is an urgent need for an effective and efficient tool or mechanism for categorizing and finding visual information on demand. Image tagging is adding text tags to an image based on what is in the image.
Although images can be annotated manually by humans, this approach can be too subjective and even impractical when there are a large number of images. However, a better approach is image auto-tagging, a time-saving and cost-effective process by which tags in the form of keywords are automatically assigned to digital images and are used to organize and retrieve images of interest from a collection or database. Images that have been auto-tagged can be retrieved using those tags or labels.
Cloudinary offers a Google auto-tagging add-on fully integrated into its image management and transformation system and allows detecting, automatically categorizing, and assigning multiple tags to images based on the categories in each image.
In this article, we’ll go through how to use Cloudinary to generate tags for images automatically, fetch the generated image tags, and access image resources based on a specified tag.
Here is a link to the demo CodeSandbox.
Create a new Next.js application using the following command:
npx create-next-app cloudinary-image-autotag
Next, let’s add the project dependencies using the following command:
npm install cloudinary axios dotenv
Now we can start our application on http://localhost:3000/ using the following command:
npm start
First, sign up for a free Cloudinary account if you don’t have one already. Displayed on your account’s Management Console (aka Dashboard) are important details: your cloud name, API key, etc.
Next, let’s create environment variables to hold the details of our Cloudinary account. Create a new file called .env at the root of your project and add the following to it:
CLOUD_NAME = YOUR CLOUD NAME HERE
API_KEY = YOUR API API KEY
API_SECRET = YOUR API API SECRET
This will be used as a default when the project is set up on another system. To update your local environment, create a copy of the .env file using the following command:
cp .env .env.local
Code language: CSS (css)By default, this local file resides in the .gitignore folder, mitigating the security risk of inadvertently exposing secret credentials to the public. You can update the .env.local file with your Cloudinary credentials.
Before using the Google Auto Tagging add-on, we need to register for the add-on. Go through these steps to register for the add-on:
- Click on the Add-ons link in your Cloudinary console.
- You should see a page consisting of all the available Cloudinary add-ons. Scroll down to locate the Google Auto Tagging add-on, click on it and select your preferred plan. We’ll be using the free plan for this project, which gives us 50 free image categorizations monthly.
So far, we’ve set up our project and registered for Cloudinary’s Google auto-tagging add-on. Let’s move on to the procedures for implementing the add-on and fetching image resources based on specified tags.
Cloudinary introduces a straightforward and uncomplicated approach to automatically generating tags or labels for images. We can generate image tags by simply setting the categorization parameter of Cloudinary’s upload method to google-tagging when calling Cloudinary’s upload method.
The uploaded image is processed and analyzed to categorize its many segments and scenes into tags automatically. The API gives a response consisting of the categorization information, including the status, individual tags, and confidence score.
The confidence score is a numerical value between 0 and 1 that specifies the discovered tag’s confidence level. The higher the level of confidence, the more confident the add-on is about the discovered tag and vice versa.
Before we begin, let’s add some styles to give our application a decent look. Add the following to your Home.module.css file in the /styles directory:
.container {
width: 1200px;
max-width: 90%;
margin: 2rem auto;
}
.container > form {
display: flex;
flex-direction: column;
align-items: center;
width: 20rem;
margin: 0 auto;
}
.container > form > button {
padding: 0.8rem 1.5rem;
background: #0000ff;
border: none;
margin: 1rem 0;
cursor: pointer;
width: 100%;
color: #fff;
font-size: 0.9rem;
font-weight: 700;
}
.tags {
display: flex;
flex-wrap: wrap;
}
.tags > button {
padding: 0.2rem 0.5rem;
margin: 0.3rem 0.2rem;
font-size: 0.8rem;
background: rgb(13, 150, 155);
border: none;
cursor: pointer;
color: #fff;
}
.images {
display: grid;
grid-template-columns: repeat(auto-fill, minmax(12rem, 1fr));
grid-gap: 0.5rem;
margin: 1.5rem 0;
}
.images > div {
width: 100%;
height: 12rem;
}
.images > div > img {
object-fit: cover;
width: 100%;
height: 100%;
}
Code language: CSS (css)Clear the existing content in your index.js file and update it with the following:
import { useState } from "react";
import styles from "../styles/Home.module.css";
import axios from "axios";
export default function Home() {
const [image, setImage] = useState("");
const handleInputOnChange = (event) => {
const reader = new FileReader();
reader.readAsDataURL(event.target.files[0]);
reader.onload = function (e) {
setImage(e.target.result);
};
};
const handleOnFormSubmit = async (event) => {
event.preventDefault();
try {
const response = await axios.post("/api/uploadImage", {
image: JSON.stringify(image),
});
console.log(response.data);
} catch (error) {
console.log("imageUpload" + error);
}
};
return (
<div className={styles.container}>
<form onSubmit={handleOnFormSubmit}>
<input type="file" onChange={handleInputOnChange}></input>
<button disabled={image ? false : true}>Upload to Cloudinary</button>
</form>
</div>
);
}
Code language: JavaScript (javascript)In the code snippet above, we made some imports, and then we created a Home component that has an image state which contains the data sent to our serverless function. The Home component renders a simple form with a file input field and a submit button. The input field has an onChange prop that triggers the handleInputOnChange function whenever a file is selected.
The handleInputOnChange function uses the File reader API to convert the selected image file into its base64 string equivalent, which is then used to update the image state on successful completion. When the submit button is clicked, the handleOnFormSubmit function gets called. The handleOnFormSubmit function makes an HTTP request to the UploadImage serverless endpoint while attaching the base64 image data payload.
With that, we’ve created the upload interface. Now, let’s create our serverless function to process the image using Cloudinary. Create a file named uploadImage.js in the /pages/api directory and add the following to it:
const cloudinary = require("cloudinary").v2;
require("dotenv").config();
cloudinary.config({
cloud_name: process.env.CLOUD_NAME,
api_key: process.env.API_KEY,
api_secret: process.env.API_SECRET,
secure: true,
});
export default async function handler(req, res) {
const image = JSON.parse(req.body.image);
try {
const cldResponse = await cloudinary.uploader.upload(image, {
categorization: "google_tagging",
auto_tagging: 0.6,
});
res.status(200).json(cldResponse);
} catch (error) {
console.log(error);
res.json({ message: "an error occured while uploading image" });
}
}
Code language: JavaScript (javascript)In the code above, we import Cloudinary and configure it with an object consisting of our Cloudinary credentials. Then we created our serverless function, which returns the response of calling Cloudinary’s upload method on success or an error if it fails.
The upload method takes the image data sent as a payload from the frontend and an object containing optional parameters as arguments. In our case, we specified google-tagging as the value of the categorization add-on, which indicates automatically generating tags using the Google auto-tagging add-on to classify the scenes of the uploaded asset.
The object also takes an optional auto_tagging parameter which defines the minimum confidence score of a detected category that should be automatically used as an assigned resource tag. For example, we specified that the add-on should only generate tags with a minimum confidence score of 0.6.
Now you can upload an image to Cloudinary and view the uploaded image and its corresponding auto-generated tags in your Cloudinary account.
We need to devise a way to fetch and display all uploaded image resources and filter through the auto-generated tags. To do that, let’s create a getImages.js file in our /pages/api folder and add the following to it:
const cloudinary = require("cloudinary").v2;
cloudinary.config({
cloud_name: process.env.CLOUD_NAME,
api_key: process.env.API_KEY,
api_secret: process.env.API_SECRET,
secure: true,
});
export default async function handler(req, res) {
try {
const images = await cloudinary.api.resources();
res.status(200).json(images.resources);
} catch (error) {
res.json({ message: "an error occured" });
}
}
Code language: JavaScript (javascript)We imported Cloudinary and configured it by passing in an object consisting of our Cloudinary credentials and created a serverless function that fetches a list of all resources from the Cloudinary account and returns it as JSON data.
Now let’s update our index.js file with the following:
//import useEffect
import { useState, useEffect } from "react";
import styles from "../styles/Home.module.css";
import axios from "axios";
export default function Home() {
const [image, setImage] = useState("");
const [cldImages, setCldImages] = useState([]);
useEffect(() => {
getAllImages();
}, []);
async function getAllImages() {
try {
const images = await axios.get("/api/getImages");
setCldImages(images.data);
} catch (error) {
console.log(error);
}
}
const handleInputOnChange = (event) => {
//...
};
const handleOnFormSubmit = async (event) => {
//...
};
return (
<div className={styles.container}>
//...
{/* Add this */}
<main>
<div className={styles.images}>
{cldImages
? cldImages.map((img) => (
<div key={img.public_id}>
<img src={img.secure_url}></img>
</div>
))
: "Loading..."}
</div>
</main>
</div>
);
}
Code language: JavaScript (javascript)We defined a new state variable, cldImages, to hold an array of images fetched from Cloudinary. Then we called a getAllImages function in the useEffect hook which initiates an axios call to the getImages serverless endpoint and updates the cldImages state with the returned value. Finally, we display a grid of the images.
So far, we’ve been able to upload images, generate tags automatically, and retrieve images from our Cloudinary account. Let’s use Cloudinary’s tags API to get a list of all the tags associated with a certain resource type. In your /pages/api folder, create a file called getImageTag.js and add the following to it:
const cloudinary = require("cloudinary").v2;
cloudinary.config({
cloud_name: process.env.CLOUD_NAME,
api_key: process.env.API_KEY,
api_secret: process.env.API_SECRET,
secure: true,
});
export default async function handler(req, res) {
try {
const imageTags = await cloudinary.api.tags({
max_results: 50,
});
res.status(200).json(imageTags);
} catch (error) {
console.log(error);
res.json({ message: "an error occurred" });
}
}
Code language: JavaScript (javascript)We configured Cloudinary and called the tags API to get a list of tags from the imageTags endpoint. The API takes an optional parameter — max_results which defines the maximum number of tags to return, and we specified 50.
To render the list of tags retrieved from Cloudinary, update your index.js file with the following:
import { useState, useEffect } from "react";
import styles from "../styles/Home.module.css";
import axios from "axios";
export default function Home() {
const [image, setImage] = useState("");
const [cldImages, setCldImages] = useState([]);
// Add this
const [imageTags, setImageTags] = useState([]);
useEffect(() => {
// Add this
async function getCldImageTags() {
try {
const cldTags = await axios.get("/api/getImageTags");
setImageTags(cldTags.data.tags);
} catch (error) {
console.log("tags" + error);
}
}
getCldImageTags()
getAllImages();
}, []);
async function getAllImages() {
//...
}
const handleInputOnChange = (event) => {
//...
};
const handleOnFormSubmit = async (event) => {
//...
};
return (
<div className={styles.container}>
//...
<main>
{/* Add this */}
<div className={styles.tags}>
{imageTags
? imageTags.map((tag) => <button key={tag}>{tag}</button>)
: "upload Image"}
</div>
<div className={styles.images}>//...</div>
</main>
</div>
);
}
Code language: JavaScript (javascript)We added a state variable called imageTags to hold an array of all retrieved tags. Then we defined a function, getCldImageTags, to initiate an axios call to the getImageTags endpoint and store the retrieved tags in the state.
Next, we iterate through the image tags and render a button for each.
Finally, we want to click on a tag and retrieve all images associated with that particular tag. Cloudinary provides an api.resources_by_tag method that handles fetching image resources based on a given tag.
The method takes a required parameter, tag, which defines the category of resources to be returned and some optional parameters. To implement this, create a file called getImagesByTag.js in your /pages/api folder and add the following to it:
const cloudinary = require("cloudinary").v2;
cloudinary.config({
cloud_name: process.env.CLOUD_NAME,
api_key: process.env.API_KEY,
api_secret: process.env.API_SECRET,
secure: true,
});
export default async function handler(req, res) {
try {
const tag = JSON.parse(req.body.tag).tag;
const resources = await cloudinary.api.resources_by_tag(tag);
res.status(200).json(resources);
} catch (error) {
console.log(error);
res.json({ message: "an error occured" });
}
}
Code language: JavaScript (javascript)This file’s setup is identical to the ones we’ve defined earlier, but we now expect a tag value obtained from the request body, which is passed to the api.resources_by_tag method to retrieve the image resources based on that tag.
Finally, let’s update our index.js file with the following:
import { useState, useEffect } from "react";
import styles from "../styles/Home.module.css";
import axios from "axios";
export default function Home() {
const [image, setImage] = useState("");
const [cldImages, setCldImages] = useState([]);
const [imageTags, setImageTags] = useState([]);
useEffect(() => {
//...
getCldImageTags()
getAllImages();
}, []);
async function getAllImages() {
//...
}
const handleInputOnChange = (event) => {
//...
};
const handleOnFormSubmit = async (event) => {
//...
};
// Add this
const handleTagClick = async (tag) => {
try {
const resImages = await axios.post("/api/getImagesByTag", {
tag: JSON.stringify({
tag: tag,
}),
});
setCldImages(resImages.data.resources);
} catch (error) {
console.log("tagClick" + error);
}
};
return (
<div className={styles.container}>
<form onSubmit={handleOnFormSubmit}>
<input type="file" onChange={handleInputOnChange}></input>
<button disabled={image ? false : true}>Upload to Cloudinary</button>
</form>
<main>
<div className={styles.tags}>
{/* Add onClick event listener to the button */}
{imageTags
? imageTags.map((tag) => (
<button key={tag} onClick={() => handleTagClick(tag)}>
{tag}
</button>
))
: "upload Image"}
</div>
<div className={styles.images}>
{cldImages
? cldImages.map((img) => (
<div key={img.public_id}>
<img src={img.secure_url} alt={img.public_id}></img>
</div>
))
: "Loading..."}
</div>
</main>
</div>
);
}
Code language: JavaScript (javascript)In the code above, we added an onClick event listener to the button element that triggers the handleTagClick function when any of the buttons get clicked. The handleTagClick function makes a call to the /api/getImagesByTag endpoint while attaching the tag as payload. The response images get saved in the cldImages to re-render a new set of images based on the clicked tag.
Find the complete project here on GitHub.
Cloudinary provides a comprehensive range of image processing and analysis tools with the ability to extract semantic data from uploaded images. In this article, we looked at how to implement Cloudinary’s Google auto-tagging add-on in a simple application. We also looked at how to retrieve images by tags.
Resources You May Find Helpful
