The modern internet is plagued with explicit videos and most children of our era have unlimited access to content on the internet. One way to protect them would be to automatically detect explicit content in videos and blur it. We can achieve this using Google’s Video Intelligence API and Cloudinary. In this tutorial, we’ll take a look at how to implement this using Next.js
The final project can be viewed on Codesandbox.
You can find the full source code on my Github repository.
First things first, you need to have Node.js and NPM installed. Second, working knowledge of Javascript, Node.js, and React/Next.js is a plus.
We’re going to be using Cloudinary for media upload and storage. It’s really easy to get started and it’s free as well. Get started with a free account at Cloudinary and then navigate to the Console page. Keep note of your Cloud name API Key and API Secret. We’ll come back to them later.
The video intelligence API is an amazing API provided by Google via the Google Cloud Project. I’m going to walk you through how to create a new project and obtain the credentials. For those familiar with GCP, you can follow the quickstart guide. Create an account if you do not already have one then navigate to the project selector page. Once here, you will need to select an existing project or create a new project. Make sure that billing is enabled for the project that you create/select. Google APIs have a free tier with a monthly limit that you can get started with. Use the APIs with caution so as not to exceed your limits. Here’s how you can confirm that billing is enabled.. The next thing we need to do is enable the Video Intelligence API so that we can use it. Navigate to the Create a new service account page and select the project you created earlier. Input an appropriate name for the service account such as blur-explicit-content-with-cloudinary
.
You can leave other options as they are and create the service account. Navigate back to the service accounts dashboard and you’ll notice your newly created service account. Under the more actions button, click on Manage keys.
Click on Add key and then on Create new key
In the pop-up dialog, make sure to choose the JSON option.
Once you’re done, a .json file will be downloaded to your computer. Take note of this file’s location as we will be using it later.
We’re now ready to get coding
Before anything else, we need to create a new Next.js project. Fire up your terminal/command line and run the following command.
npx create-next-app blur-explicit-content-with-cloudinary
Code language: JavaScript (javascript)This will scaffold a basic project called blur-explicit-content-with-cloudinary. You can look at the official documentation for more advanced options such as typescript. Change the directory into the new project and open it in your favorite code editor.
cd blur-explicit-content-with-cloudinary
Code language: JavaScript (javascript)Let’s start by creating a few functions that will handle the upload to cloudinary and deletion of media.
We need to install the Cloudinary SDK first
npm install --save cloudinary
Next, create a folder called lib/ at the root of your project. Create a new file called cloudinary.js and paste the following code inside.
// lib/cloudinary.js
// Import the v2 api and rename it to cloudinary
import { v2 as cloudinary } from 'cloudinary';
// Initialize the SDK with cloud_name, api_key, and api_secret
cloudinary.config({
cloud_name: process.env.CLOUD_NAME,
api_key: process.env.API_KEY,
api_secret: process.env.API_SECRET,
});
const FOLDER_NAME = 'explicit-videos/';
export const handleCloudinaryUpload = (path, transformation = []) => {
// Create and return a new Promise
return new Promise((resolve, reject) => {
// Use the sdk to upload media
cloudinary.uploader.upload(
path,
{
// Folder to store video in
folder: FOLDER_NAME,
// Type of resource
resource_type: 'video',
transformation,
},
(error, result) => {
if (error) {
// Reject the promise with an error if any
return reject(error);
}
// Resolve the promise with a successful result
return resolve(result);
}
);
});
};
export const handleCloudinaryDelete = async (ids) => {
return new Promise((resolve, reject) => {
cloudinary.api.delete_resources(
ids,
{
resource_type: 'video',
},
(error, result) => {
if (error) {
return reject(error);
}
return resolve(result);
}
);
});
};
Code language: JavaScript (javascript)We first import the cloudinary v2 SDK and rename it to cloudinary. This is only for readability purposes. Next, we initialize the SDK by calling the config method on the SDK. We pass the cloud_name, api_key and api_secret. We’re using environment variables here, which we’ll be defining in a while. We also define a folder name where we’re going to be storing all of our videos. The handleCloudinaryUpload function takes in a path to the file that we want to upload, and an optional array of transformations to run on the video. Inside this function, we’re calling the uploader.upload method on the cloudinary SDK to upload the file. Read more about the upload media api and options you can pass from the official documentation. The handleCloudinaryDelete takes in an array of public IDs belonging to the resources we want to delete, then calls the api.delete_resources method on the SDK. Read more about this here. Let’s define those environment variables. Luckily, Next.js has inbuilt support for environment variables. This topic is covered in-depth in their docs. Create a file called .env.local at the root of your project and paste the following inside.
CLOUD_NAME=YOUR_CLOUD_NAME
API_KEY=YOUR_API_KEY
API_SECRET=YOUR_API_SECRET
Make sure to replace YOUR_CLOUD_NAME YOUR_API_KEY and YOUR_API_SECRET with the appropriate values that we got from the cloudinary-credentials section.
And that’s it for this file.
Let’s now create the functions that will allow us to communicate with the Video Intelligence API.
First thing is to install the dependencies.
npm install @google-cloud/video-intelligence
Code language: CSS (css)Create a new file under lib/ called google.js and paste the following code inside.
// lib/google.js
import {
VideoIntelligenceServiceClient,
protos,
} from '@google-cloud/video-intelligence';
const client = new VideoIntelligenceServiceClient({
// Google cloud platform project id
projectId: process.env.GCP_PROJECT_ID,
credentials: {
client_email: process.env.GCP_CLIENT_EMAIL,
private_key: process.env.GCP_PRIVATE_KEY.replace(/\\n/gm, '\n'),
},
});
/**
*
* @param {string | Uint8Array} inputContent
* @returns {Promise<protos.google.cloud.videointelligence.v1.VideoAnnotationResults>}
*/
export const annotateVideoWithLabels = async (inputContent) => {
// Grab the operation using array destructuring. The operation is the first object in the array.
const [operation] = await client.annotateVideo({
// Input content
inputContent: inputContent,
// Video Intelligence features
features: ['EXPLICIT_CONTENT_DETECTION'],
});
const [operationResult] = await operation.promise();
// Gets annotations for video
const [annotations] = operationResult.annotationResults;
return annotations;
};
Code language: JavaScript (javascript)We first import the VideoIntelligenceServiceClient and then proceed to create a new client. The client takes in the project id and a credentials object containing the client’s email and private key. There are many different ways of authenticating Google APIs. Have a read in the official documentation. We’ll define the environment variables that we have just used shortly. The annotateVideoWithLabels takes in a string or a buffer array and then calls the client’s annotateVideo method with a few options. Read more about these options in the official documentation. The most important is the features option. Here we need to tell Google what operation to run. In this case, we only pass the EXPLICIT_CONTENT_DETECTION. Read all about this here. We then wait for the operation to complete by calling promise() on the operation and waiting for the Promise to complete. We then get the operation result using Javascript’s destructuring. To understand the structure of the resulting data, take a look at the official documentation. We then proceed to get the first item in the annotation results and return that. And now for those environment variables. Add the following to the .env.local file we created earlier
GCP_PROJECT_ID=YOUR_GCP_PROJECT_ID
GCP_PRIVATE_KEY=YOUR_GCP_PRIVATE_KEY
GCP_CLIENT_EMAIL=YOUR_GCP_CLIENT_EMAIL
You can find YOUR_GCP_PROJECT_ID,YOUR_GCP_PRIVATE_KEY and YOUR_GCP_CLIENT_EMAIL in the .json file that we downloaded in the google-cloud-project-and-credentials section.
Now let’s move on to the slightly hard part.
We’ll be using Next.js API routes to trigger the video upload. Read more about API routes in the official docs. Create a file called videos.js under the pages/api/ folder and paste the following code inside.
// pages/api/videos.js
// Next.js API route support: https://nextjs.org/docs/api-routes/introduction
import { promises as fs } from 'fs';
import { annotateVideoWithLabels } from '../../lib/google';
import {
handleCloudinaryDelete,
handleCloudinaryUpload,
} from '../../lib/cloudinary';
import { createWriteStream, promises } from 'fs';
import { get } from 'https';
const videosController = async (req, res) => {
// Check the incoming HTTP method. Handle the POST request method and reject the rest.
switch (req.method) {
// Handle the POST request method
case 'POST': {
try {
const result = await handlePostRequest();
// Respond to the request with a status code 201(Created)
return res.status(201).json({
message: 'Success',
result,
});
} catch (error) {
// In case of an error, respond to the request with a status code 400(Bad Request)
return res.status(400).json({
message: 'Error',
error,
});
}
}
// Reject other http methods with a status code 405
default: {
return res.status(405).json({ message: 'Method Not Allowed' });
}
}
};
const handlePostRequest = async () => {
// Path to the file you want to upload
const pathToFile = 'public/videos/explicit.mp4';
// Read the file using fs. This results in a Buffer
const file = await fs.readFile(pathToFile);
// Convert the file to a base64 string in preparation for analyzing the video with google's video intelligence api
const inputContent = file.toString('base64');
// Analyze the video using google video intelligence api and annotate explicit frames
const annotations = await annotateVideoWithLabels(inputContent);
// Group all adjacent frames with the same pornography likelihood
const likelihoodClusters = annotations.explicitAnnotation.frames.reduce(
(prev, curr) => {
if (
prev.length &&
curr.pornographyLikelihood ===
prev[prev.length - 1][0].pornographyLikelihood
) {
prev[prev.length - 1].push(curr);
} else {
prev.push([curr]);
}
return prev;
},
[]
);
// Get the frames with a pornography likelihood greater than 2
const likelyFrames = likelihoodClusters.filter((cluster) =>
cluster.some((frame) => frame.pornographyLikelihood > 2)
);
// Set the start offset for the main explicit video
let initialStartOffset = 0;
// Array to hold all uploaded videos
const uploadResults = [];
// Loop through the frames with a pornography likelihood greater than 2
for (const likelyFrame of likelyFrames) {
// Get the start offset of the segment
const startOffset =
parseInt(likelyFrame[0].timeOffset.seconds ?? 0) +
(likelyFrame[0].timeOffset.nanos ?? 0) / 1000000000;
// Get the end offset of the segment
const endOffset =
parseInt(likelyFrame[likelyFrame.length - 1].timeOffset.seconds ?? 0) +
(likelyFrame[likelyFrame.length - 1].timeOffset.nanos ?? 0) / 1000000000 +
0.1;
let unlikelyFrameUploadResult;
if (startOffset != 0) {
// This will upload the segment that is clean and doesn't need any blurring
unlikelyFrameUploadResult = await handleCloudinaryUpload(pathToFile, [
{ offset: [initialStartOffset, startOffset] },
]);
}
// Upload the explicit segment to cloudinary and apply a blur effect
const uploadResult = await handleCloudinaryUpload(pathToFile, [
{ offset: [startOffset, endOffset], effect: 'blur:1500' },
]);
// Push the upload result for the segment that doesn't need to be blurred and the segment next to it that has been blurred.
uploadResults.push(
{
startOffset: initialStartOffset,
endOffset: startOffset,
uploadResult: unlikelyFrameUploadResult,
},
{ startOffset, endOffset, uploadResult }
);
initialStartOffset = endOffset;
}
// Upload the last segment to cloudinary if any
const uploadResult = await handleCloudinaryUpload(pathToFile, [
{ start_offset: initialStartOffset },
]);
uploadResults.push({
startOffset: initialStartOffset,
endOffset: null,
uploadResult,
});
const firstFilePath = await downloadVideo(
uploadResults[0].uploadResult.secure_url,
uploadResults[0].uploadResult.public_id.replace(/\//g, '-')
);
const fullVideoUploadResult = await handleCloudinaryUpload(firstFilePath, [
uploadResults.slice(1).map((video) => ({
flags: 'splice',
overlay: `video:${video.uploadResult.public_id.replace(/\//g, ':')}`,
})),
]);
await handleCloudinaryDelete([
uploadResults.map((video) => video.uploadResult.public_id),
]);
return {
uploadResult: fullVideoUploadResult,
};
};
const downloadVideo = (url, name) => {
return new Promise((resolve, reject) => {
try {
get(url, async (res) => {
const downloadPath = `public/videos/downloads`;
await promises.mkdir(downloadPath, { recursive: true });
const filePath = `${downloadPath}/${name}.mp4`;
const file = createWriteStream(filePath);
res.pipe(file);
res.on('error', (error) => {
reject(error);
});
file.on('error', (error) => {
reject(error);
});
file.on('finish', () => {
file.close();
resolve(file.path);
});
});
} catch (error) {
reject(error);
}
});
};
export default videosController;
Code language: JavaScript (javascript)The videosController function is what handles the API request. We’ll only handle the POST requests and return a response of status code 405 – Method not allowed for all other request types.
In the handlePostRequest function, we first define the path to the file that we want to be analyzed. Now, for a real-world app, you would want to upload a video from the user’s browser and analyze that. For the sake of simplicity, we’re using a static path. The variable pathToFile holds a path that points to the video that we want to analyze. If you’d like to use the same video I used, just clone the full project from my Github and you can find it in the public/videos folder.
We convert the video file to a base64 string using file.toString("base64") and then call the annotateVideoWithLabels function that we created earlier. Google Video Intelligence annotates the video frame by frame instead of in segments. We need a way to group adjacent frames that have the same pornography likelihood. This is done in the following piece of code.
// Group all adjacent frames with the same pornography likelyhood
const likelihoodClusters = annotations.explicitAnnotation.frames.reduce(
(prev, curr) => {
if (
prev.length &&
curr.pornographyLikelihood ===
prev[prev.length - 1][0].pornographyLikelihood
) {
prev[prev.length - 1].push(curr);
} else {
prev.push([curr]);
}
return prev;
},
[]
);
Code language: JavaScript (javascript)Once we have that we filter to only get the frames that have a pornography likelihood higher than 2. There are six levels of likelihood. See here. We only want to match the frames that are either possible, likely, or very likely. This is done in the following piece of code.
// Get the frames with a pornogrphy likelihood greater than 2
const likelyFrames = likelihoodClusters.filter((cluster) =>
cluster.some((frame) => frame.pornographyLikelihood > 2)
);
Code language: JavaScript (javascript)The next thing will be to iterate through the matched frame clusters. We’ll get each cluster’s first frame and last frames to get the start offset and end offset respectively. We’ll cut each segment from the main video and upload that to cloudinary. We also apply a blur effect to each segment. This is all done by the transformations that we pass to the handleCloudinaryUpload function.
// Upload the frame to cloudinary and apply a blur effect
const uploadResult = await handleCloudinaryUpload(pathToFile, [
{ offset: [startOffset, endOffset], effect: "blur:1500" },
]);
Code language: JavaScript (javascript)Read more about the transformations here.
We push the upload result for each segment to an array called uploadResults so that we can join them all together later. Finally, we download the very first segment that we uploaded using the downloadVideo function and then concatenate all the other segments to it. With that we now have a full video, we’ll return the result of that.
The downloadVideo function is self-explanatory. It just gets the file using the get method from the https package and saves it in the public/videos/downloads folder. We’re done with the backend. Let’s move on to the front end.
Paste the following inside pages/index.js.
// pages/index.js
import Head from 'next/head';
import { useState } from 'react';
export default function Home() {
const [video, setVideo] = useState(null);
const [loading, setLoading] = useState(false);
const handleUploadVideo = async () => {
try {
// Set loading to true
setLoading(true);
// Make a POST request to the `api/videos/` endpoint
const response = await fetch('/api/videos', {
method: 'post',
});
const data = await response.json();
// Check if the response is successful
if (response.status >= 200 && response.status < 300) {
const result = data.result;
// Update our videos state with the results
setVideo(result);
} else {
throw data;
}
} catch (error) {
// TODO: Handle error
console.error(error);
} finally {
setLoading(false);
// Set loading to true once a response is available
}
};
return (
<div>
<Head>
<title>
{' '}
Blur explicit content with Google Video Intelligence and Cloudinary
</title>
<meta
name='description'
content=' Blur explicit content with Google Video Intelligence and Cloudinary'
/>
<link rel='icon' href='/favicon.ico' />
</Head>
<header>
<h1>
Blur explicit content with Google Video Intelligence and Cloudinary
</h1>
</header>
<main>
<hr />
<div className='upload-wrapper'>
<button onClick={handleUploadVideo} disabled={loading || video}>
Upload
</button>
</div>
<hr />
{loading && <div className='loading'>Loading...</div>}
{video ? (
[
<div
className='original-video-wrapper'
key='original-video-wrapper'
>
<h2>Original Video</h2>
<video src='/videos/explicit.mp4' controls></video>
</div>,
<hr key='videos-break' />,
<div className='blurred-video-wrapper' key='blurred-video-wrapper'>
<h2>Blurred Video</h2>
<video src={video.uploadResult.secure_url} controls></video>
</div>,
]
) : (
<div className='no-video'>
<p>Tap On The Upload Button To Load Video</p>
</div>
)}
</main>
<style jsx>{`
header {
width: 100%;
min-height: 100px;
display: flex;
align-items: center;
justify-content: center;
}
main {
min-height: 100vh;
}
main div.upload-wrapper {
display: flex;
justify-content: center;
align-items: center;
padding: 20px 0;
}
main div.upload-wrapper button {
padding: 10px;
min-width: 200px;
height: 50px;
}
main div.loading {
display: flex;
justify-content: center;
align-items: center;
background-color: #9900ff;
color: #ffffff;
height: 150px;
}
main div.original-video-wrapper {
width: 100%;
display: flex;
flex-flow: column;
justify-content: center;
align-items: center;
}
main div.original-video-wrapper video {
width: 80%;
}
main div.blurred-video-wrapper {
width: 100%;
display: flex;
flex-flow: column;
justify-content: center;
align-items: center;
}
main div.blurred-video-wrapper video {
width: 80%;
}
main div.no-video {
background-color: #ececec;
min-height: 300px;
display: flex;
flex-flow: column;
justify-content: center;
align-items: center;
}
`}</style>
</div>
);
}
Code language: JavaScript (javascript)This is just standard React. The handleUploadVideo function makes a POST request to the /api/videos/ endpoint that we created earlier and updates the video state with the result. For the html, we just have an upload button that will trigger the handleUploadVideo, we also have two video elements, one for the original video and another for the blurred video. The rest is just some CSS.
With this, you’re ready to run your project.
npm run dev