Amazon Rekognition Video Moderation

Cloudinary is a cloud-based service that provides an end-to-end media asset management solution including uploads, storage, manipulations, optimizations and delivery. Cloudinary offers a very rich set of manipulation and analysis capabilities, and allows you to upload images and videos to the cloud, manipulate them on-the-fly and deliver them to your users optimized and cached via a fast CDN.

Amazon Rekognition is a service that makes it easy to add video analysis to your applications. Cloudinary provides an add-on for Amazon Rekognition's video moderation service based on deep learning algorithms, fully integrated into Cloudinary's video management and manipulation pipeline.

With Amazon Rekognition's AI Moderation add-on, you can extend Cloudinary's powerful cloud-based media library and delivery capabilities with automatic artificial intelligence based moderation of your videos. Protect your users from explicit and suggestive adult content in your user uploaded videos, making sure that no offensive videos are displayed to your web and mobile viewers.

Getting started

Before you can use the Amazon Rekognition Video Moderation add-on:

  • You must have a Cloudinary account. If you don't already have one, you can sign up for a free account.

  • Register for the add-on: make sure you're logged in to your account and then go to the Add-ons page. For more information about add-on registrations, see Registering for add-ons.

  • Keep in mind that many of the examples on this page use our SDKs. For SDK installation and configuration details, see the relevant SDK guide.

  • If you are new to Cloudinary, you may want to take a look at How to integrate Cloudinary in your app for a walk through on the basics of creating and setting up your account, working with SDKs, and then uploading, transforming and delivering assets.

Automatic video moderation flow

The following list describes the flow of uploading and displaying moderated videos using Cloudinary and the Amazon Rekognition's AI Moderation add-on:

  • Your users upload a video to Cloudinary through your application.
  • The uploaded video is sent to Amazon Rekognition for moderation.
  • The video is either approved or rejected by Amazon Rekognition.
  • An optional notification callback is sent to your application with the video moderation result.
  • A rejected video is moved to a secondary backup repository, consuming additional storage.
  • Moderated videos can be listed programmatically using Cloudinary's Admin API or interactively using our online Media Library Web interface.
  • You can manually override the automatic moderation using the Admin API or the Media Library.

Moderation categorization

Amazon Rekognition assigns a moderation confidence score (0 - 100) indicating the chances that a video belongs to an offensive content category.

There are two levels of categories for labelling unsafe content, with each top-level category containing a number of second-level categories, for example under the 'Violence' (violence) category you have the sub-category 'Physical Violence' (physical_violence). A full list of all the latest available categories and sub-categories is provided by AWS, see Amazon Rekognition categories.

The top level categories include:

  • Explicit Nudity (explicit_nudity)
  • Suggestive (suggestive)
  • Violence (violence)
  • Visually Disturbing (visually_disturbing)
  • Rude Gestures (rude_gestures)
  • Drugs (drugs)
  • Tobacco (tobacco)
  • Alcohol (alcohol)
  • Gambling (gambling)
  • Hate Symbols (hate_symbols)

Note
When referring to the categories in your code, replace spaces with underscores and upper case letters with lower case letters (e.g., 'Illustrated Nudity Or Sexual Activity' becomes illustrated_nudity_or_sexual_activity).

The default moderation confidence level to reject a video is 0.5 for all categories, unless specifically overridden (see explanation and examples below). All videos classified by Amazon Rekognition with a value greater than the moderation confidence level (in any of the categories) are classified as 'rejected', otherwise their status is set to 'approved'.

Request video moderation

In order to request moderation while uploading a video, with default moderation confidence levels, simply set the moderation upload API parameter to aws_rek_video:

Ruby:
Copy to clipboard
Cloudinary::Uploader.upload("my_file.mp4",
  :resource_type => "video", 
  :moderation => "aws_rek_video")
PHP:
Copy to clipboard
\Cloudinary\Uploader::upload("my_file.mp4", 
  array(
    "resource_type" => "video", 
    "moderation" => "aws_rek_video"));
Python:
Copy to clipboard
cloudinary.uploader.upload("my_file.mp4",
  resource_type = "video", 
  moderation = "aws_rek_video")
Node.js:
Copy to clipboard
cloudinary.v2.uploader.upload("my_file.mp4", 
  { resource_type: "video", 
    moderation: "aws_rek_video"},
  function(error, result){console.log(result);});
Java:
Copy to clipboard
cloudinary.uploader().upload("my_file.mp4", 
  ObjectUtils.asMap(
    "resource_type", "video", 
    "moderation", "aws_rek_video"));
.NET:
Copy to clipboard
var uploadParams = new VideoUploadParams(){
  File = new FileDescription(@"my_file.mp4"),
  Moderation = "aws_rek_video"};
var uploadResult = cloudinary.Upload(uploadParams);
iOS:
Copy to clipboard
let params = CLDUploadRequestParams()
  .setResourceType(.video)
  .setModeration("aws_rek_video")
var mySig = MyFunction(params)  // your own function that returns a signature generated on your backend
params.setSignature(CLDSignature(signature: mySig.signature, timestamp: mySig.timestamp))
let request = cloudinary.createUploader().signedUpload(
  url: "my_file.mp4", params: params)

Tip
You can use upload presets to centrally define a set of upload options including add-on operations to apply, instead of specifying them in each upload call. You can define multiple upload presets, and apply different presets in different upload scenarios. You can create new upload presets in the Upload page of the Management Console settings or using the upload_presets Admin API method. From the Upload page of the console settings, you can also select default upload presets to use for image, video, and raw API uploads (respectively) as well as default presets for image, video, and raw uploads performed via the Media Library UI.

Learn more: Upload presets

You can also override the default moderation confidence level of 0.5 on a per category basis by including the category name and new value as part of the 'moderation' parameter value, separated by colons. You can override multiple categories by adding them in series together with their new value, all separated by colons.

Notes

  • The confidence level must be provided as a decimal number between 0.0 and 1.0.
  • Videos must have a minimum height and width of 80 pixels.
  • Overriding the default moderation confidence value of a top-level category will also set all its child categories to the same value.
  • To exclude a category from the moderation check, set the category's value to ignore.
  • To return the moderation_labels array, even if no offending content is found, you can set aws_rek:min_confidence:0.0.

For example, to request moderation with the moderation confidence level set to 0.85 for the 'Female Swimwear or Underwear' child category, 0.7 for the 'Explicit Nudity' top-level category (overriding the default for all its child categories as well) and exclude the 'Revealing Clothes' category:

Ruby:
Copy to clipboard
Cloudinary::Uploader.upload("my_file.mp4",
  :resource_type => "video",  
  :moderation => "aws_rek_video:female_swimwear_or_underwear:0.85:explicit_nudity:0.7:revealing_clothes:ignore")
PHP:
Copy to clipboard
\Cloudinary\Uploader::upload("my_file.mp4", 
  array(
    "resource_type" => "video", 
    "moderation" => "aws_rek_video:female_swimwear_or_underwear:0.85:explicit_nudity:0.7:revealing_clothes:ignore"));
Python:
Copy to clipboard
cloudinary.uploader.upload("my_file.mp4",
  resource_type = "video", 
  moderation = "aws_rek_video:female_swimwear_or_underwear:0.85:explicit_nudity:0.7:revealing_clothes:ignore")
Node.js:
Copy to clipboard
cloudinary.v2.uploader.upload("my_file.mp4", 
  { resource_type: "video", 
    moderation: "aws_rek_video:female_swimwear_or_underwear:0.85:explicit_nudity:0.7:revealing_clothes:ignore"},
  function(error, result){console.log(result);});
Java:
Copy to clipboard
cloudinary.uploader().upload("my_file.mp4", 
  ObjectUtils.asMap(
    "resource_type", "video", 
    "moderation", "aws_rek_video:female_swimwear_or_underwear:0.85:explicit_nudity:0.7:revealing_clothes:ignore"));
.NET:
Copy to clipboard
var uploadParams = new VideoUploadParams(){
  File = new FileDescription(@"my_file.mp4"),
  Moderation = "aws_rek_video:female_swimwear_or_underwear:0.85:explicit_nudity:0.7:revealing_clothes:ignore"};
var uploadResult = cloudinary.Upload(uploadParams);
iOS:
Copy to clipboard
let params = CLDUploadRequestParams()
  .setResourceType(.video)
  .setModeration("aws_rek_video:female_swimwear_or_underwear:0.85:explicit_nudity:0.7:revealing_clothes:ignore")
var mySig = MyFunction(params)  // your own function that returns a signature generated on your backend
params.setSignature(CLDSignature(signature: mySig.signature, timestamp: mySig.timestamp))
let request = cloudinary.createUploader().signedUpload(
  url: "my_file.mp4", params: params)

Moderation response

As moderation is processed asynchronously, the initial upload response will include a moderation parameter indicating the status and kind of moderation:

Copy to clipboard
"moderation":[{ "status":"pending", "kind": "aws_rek_video" }]

Once the moderation is complete, you will receive a notification indicating the results of the request for moderation. The snippet below shows that the moderation result has put the video in 'rejected' status.

Copy to clipboard
{
...
    "moderation_response": {
        "moderation_labels": [
            {
                "timestamp": 2999,
                "moderation_label": {
                    "confidence": 65.07359313964844,
                    "name": "Sexual Activity",
                    "parent_name": "Explicit Nudity"
                }
            },
            {
                "timestamp": 3199,
                "moderation_label": {
                    "confidence": 62.840023040771484,
                    "name": "Explicit Nudity",
                    "parent_name": ""
                }
            },
            {
                "timestamp": 3199,
                "moderation_label": {
                    "confidence": 59.84464645385742,
                    "name": "Sexual Activity",
                    "parent_name": "Explicit Nudity"
                }
            },
            {
                "timestamp": 3399,
                "moderation_label": {
                    "confidence": 50.517578125,
                    "name": "Explicit Nudity",
                    "parent_name": ""
                }
            },
            {
                "timestamp": 6600,
                "moderation_label": {
                    "confidence": 65.33306121826172,
                    "name": "Explicit Nudity",
                    "parent_name": ""
                }
            },
            {
                "timestamp": 6600,
                "moderation_label": {
                    "confidence": 57.57171630859375,
                    "name": "Sexual Activity",
                    "parent_name": "Explicit Nudity"
                }
            },
            {
                "timestamp": 6800,
                "moderation_label": {
                    "confidence": 80.53067779541016,
                    "name": "Explicit Nudity",
                    "parent_name": ""
                }
            },
            {
                "timestamp": 6800,
                "moderation_label": {
                    "confidence": 76.79461669921875,
                    "name": "Sexual Activity",
                    "parent_name": "Explicit Nudity"
                }
            }
        ],
        "moderation_status": "rejected",
        "moderation_kind": "aws_rek_video",
        "moderation_updated_at": "2020-03-02T09:33:43Z"
    }
...
}

Note
Only the first 1000 moderation_labels are returned in the response but all of them are taken into account for moderation.

Video moderation listing

Cloudinary's Admin API can be used to list all moderated videos. You can list either the approved or the rejected videos by specifying the second parameter of the resources_by_moderation API method. For example to list all rejected videos:

Ruby:
Copy to clipboard
Cloudinary::Api.resources_by_moderation("aws_rek_video", "rejected", :resource_type => "video")
PHP:
Copy to clipboard
$api->resources_by_moderation("aws_rek_video", "rejected", array("resource_type" => "video"));
Python:
Copy to clipboard
cloudinary.api.resources_by_moderation("aws_rek_video", "rejected", resource_type = "video")
Node.js:
Copy to clipboard
cloudinary.api.resources_by_moderation("aws_rek_video", "rejected", { resource_type: "video" },
  function(error, result) { console.log(result); });
Java:
Copy to clipboard
cloudinary.api().resourcesByModeration("aws_rek_video", "rejected", 
  ObjectUtils.emptyMap("resource_type", "video"));
.NET:
Copy to clipboard
var listResourcesResult = cloudinary.ListResourcesByModerationStatus("aws_rek_video", "rejected");
Copy to clipboard
{
 "resources": 
  [{
    "public_id": "q7vcvrfjm9mj4bfp3qc8",
    "format": "mp4",
    "version": 1570976812,
    "resource_type": "video",
    "type": "upload",
    "created_at": "2017-05-04T41:02:23Z",
    "bytes": 120253,
    "width": 864,
    "height": 576,
    "backup": true,
    "url": "http://res.cloudinary.com/demo/video/upload/v1570976812/q7vcvrfjm9mj4bfp3qc8.mp4",
    "secure_url": "https://res.cloudinary.com/demo/video/upload/v1570976812/q7vcvrfjm9mj4bfp3qc8.mp4"
  },
  {
    "public_id": "zp4fgdbabhlwef7bxu84",
    "format": "mp4",
    ...
  }
  ...
 ]
}

Manual override

As the automatic video moderation of the Amazon Rekognition AI Moderation add-on is based on a decision made by an advanced algorithm, in some cases you may want to manually override the moderation decision and either approve a previously rejected video or reject an approved one.

Overriding moderation via the Media Library

One way to manually override the moderation result is using Cloudinary's Media Library Web interface. From the left navigation menu, select Moderation. Then, from moderation tools list in the top menu, select Rekognition and then select which moderation queue of videos to display (Rejected, or Approved).

  • When listing the queue of videos rejected by Amazon Rekognition, you can click on the green Approve button to revert the decision and recover the original rejected video.
  • When listing the queue of videos approved by Amazon Rekognition, you can click on the red Reject button to revert the decision and prevent a certain video from being publicly available to your users.

Overriding moderation via the Admin API

Alternatively to using the Media Library interface, you can use Cloudinary's Admin API to manually override the moderation result. The following example uses the update API method while specifying a public ID of a moderated video and setting the moderation_status parameter to the rejected status.

Ruby:
Copy to clipboard
Cloudinary::Api.update("hwepb67oxzh4lrigssld", 
  :resource_type => "video", 
  :moderation_status => "rejected")
PHP:
Copy to clipboard
$api->update("hwepb67oxzh4lrigssld", 
  array(
    "resource_type" => "video", 
    "moderation_status" => "rejected"));
Python:
Copy to clipboard
cloudinary.api.update("hwepb67oxzh4lrigssld",
  resource_type = "video", 
  moderation_status = "rejected")
Node.js:
Copy to clipboard
cloudinary.v2.api.update("hwepb67oxzh4lrigssld",
  { resource_type: "video", 
    moderation_status: "rejected" },
  function(error, result){console.log(result);});
Java:
Copy to clipboard
cloudinary.api().update("hwepb67oxzh4lrigssld", 
  ObjectUtils.asMap(
    "resource_type", "video", 
    "moderation_status", "rejected"));
.NET:
Copy to clipboard
var updateParams = new UpdateParams("hwepb67oxzh4lrigssld"){
  ResourceType = "video",
  ModerationStatus = "rejected"};
var updateResult = cloudinary.Update(updateParams);

✔️ Feedback sent!

Rate this page: