Cloudinary Blog

Automatic image moderation, removing adult or inappropriate photos using WebPurify and Cloudinary

Automatic Image Moderation with Minimal Developer Effort

If your web or mobile application involves user-generated content, you may encounter users who upload inappropriate photos or images to your application. These could be images which offend other users - adult content, violent photos, etc. - or images which cause your site to violate laws or regulations.

There are two ways to identify and remove such images: you can either require approval of each image before it is displayed to your users, or display images immediately after upload, and then quickly remove them from your site as soon as a moderator has found them to be inappropriate.

Another issue is who does the moderation. You can hire and train internal personnel to manually moderate your images. In Cloudinary's image management solution, we provide a manual moderation web interface and API to help do this efficiently. However, manual moderation is time consuming and expensive, and so we wanted to provide an additional option - automatic moderation of images as your users upload them.

Introducing Cloudinary's WebPurity add-on - seamless automatic image moderation

WebPurify logo WebPurify is an online service for automatically moderating and filtering out inappropriate photos, preventing adult content or other kinds of offensive photos from creeping into your website. WebPurify performs quick and thorough human review of your user-uploaded images, behind the scenes.

Cloudinary can manage your entire image lifecycle - upload and cloud storage of images, manipulating images to match your graphic design, and delivery of images to users over a fast CDN. With our new WebPurify Image Moderation add-on, you can also use Cloudinary to seamlessly perform automatic moderation of images uploaded by your users. Rejected images are automatically removed from your site, and you will be notified on every reject.

Enabling automatic image moderation by WebPurify within Cloudinary

Sign up to a Cloudinary account if you don't have one already. Subscribe to the WebPurify add-on, free tier available.

WebPurify Cloudinary Add-on

Cloudinary provides an image upload API. Simply set the moderation parameter to webpurify when calling Cloudinary's upload API, as you can see in the following code samples.

Ruby:
Cloudinary::Uploader.upload("local_file.jpg", :moderation => "webpurify")
PHP:
\Cloudinary\Uploader::upload("local_file.jpg", array("moderation" => "webpurify"));
Python:
cloudinary.uploader.upload("local_file.jpg", moderation = "webpurify")
Node.js:
cloudinary.uploader.upload("local_file.jpg", 
  function(result) { console.log(result); }, { moderation: "webpurify" });
Java:
cloudinary.uploader().upload("local_file.jpg", 
  Cloudinary.asMap("moderation", "webpurify"));

The uploaded image is marked as 'pending moderation' and is automatically sent to WebPurify. Moderation is performed asynchronously after the upload call is completed. You can decide whether to display the image to your users immediately, or wait for moderation approval. If you decide not to wait, the image is delivered with short-term cache settings, allowing fast removal of rejected images.

Note - Cloudinary delivers images using a fast CDN (images are cached close to the geographical location of your users). While an image is pending moderation, the cache will be set to 'short-term', so that if it's rejected in moderation, it will be easy to roll back and remove it from all cache servers.

If you are already using Cloudinary for user-uploaded images, simply subscribe to the WebPurify add-on and set the moderation parameter as above. It's that easy to ensure all the images on your dynamic site are safe for users and comply with relevant laws.

Handling moderation results

If an uploaded photo is approved by the WebPurify add-on, it is marked as 'approved' in your cloud-based media library managed by Cloudinary. Cloudinary updates the approved images, and all their transformed versions (Cloudinary can generate thumbnails, resized versions, and many other on-the-fly image manipulations) to long term caching. This optimizes delivery of the images to users around the world, now that there is no longer a need to “roll back” the image in case they are rejected.

When an uploaded photo is rejected by the WebPurify add-on, because it includes inappropriate content, the image is automatically deleted from your media library, all delivered versions of the rejected image expire quickly, and the rejected image is no longer displayed. The original rejected images are stored in the backup repository of your Cloudinary account, which you can still access through our web interface.

You may want to automatically update your application when an image is rejected by the WebPurify add-on. In order to do that, you can set the notification_url upload parameter to a URL on your site:

Ruby:
Cloudinary::Uploader.upload("local_file.jpg", 
  :moderation => "webpurify",
  :notification_url => "http://mysite.example.com/hooks")
PHP:
\Cloudinary\Uploader::upload("local_file.jpg", 
  array(
    "moderation" => "webpurify",
    "notification_url" => "http://mysite.example.com/hooks"));
Python:
cloudinary.uploader.upload("local_file.jpg",
  moderation = "webpurify",
  notification_url = "http://mysite.example.com/hooks")
Node.js:
cloudinary.uploader.upload("local_file.jpg", 
  function(result) { console.log(result); }, 
  { 
    moderation: "webpurify",
    notification_url: "http://mysite.example.com/hooks" 
});
Java:
cloudinary.uploader().upload("local_file.jpg", 
  Cloudinary.asMap("moderation", "webpurify", 
    "notification_url", "http://mysite.example.com/hooks"));

An HTTP callback request will be sent to your servers when the moderation process is completed (either approved or rejected). The request data includes the identifier and URLs of the moderated images.

{
 "moderation_response": "rejected",
 "moderation_status": "rejected",
 "moderation_kind": "webpurify",
 "moderation_updated_at": "2014-03-02T20:47:48Z",
 "public_id": "nuzn4riqxhhzfyfljjxv",
 "uploaded_at": "2014-03-02T20:47:47Z",
 "version": 1393793267,
 "url":
 "https://res.cloudinary.com/demo/image/upload/v1393793267/nuzn4riqxhhzfyfljjxv.jpg",
 "secure_url":
 "https://res.cloudinary.com/demo/image/upload/v1393793267/nuzn4riqxhhzfyfljjxv.jpg",
 "etag":"06778590d96907b60b5fa83795e7df3b",
 "notification_type":"moderation"
}    

In addition, Cloudinary provides tools for further managing the image moderation flow. You can use Cloudinary's Admin API to list the queue of images pending moderation, and list rejected or approved images. Or you can view these lists using Cloudinary's Media Library web interface.

You can also use the API or web interface to alter the automatic moderation decision. You can browse rejected or approved images, and manually approve or reject them.

WebPurify moderation - media library

If you choose to approve a previously rejected image, the original version of the rejected image will be restored from backup. If you choose to reject a previously approved image, cache invalidation will be performed for the image so it is gradually erased from all the CDN cache servers.

For more details about the various moderation management APIs, take a look at our detailed documentation of the WebPurify image moderation add-on.

Summary

Websites enabling user-uploaded content have become very common. Most web and mobile apps skip moderation and take the chance that adult or inappropriate content might be uploaded and displayed to their users. Some apps support user reporting of problematic content, but this might be too late in certain scenarios, and the reporting feature can be abused as well.

Cloudinary takes care of the entire image lifecycle - uploading images to the cloud, dynamically manipulating them to match your graphic design, and delivering them to users optimized and cached via fast CDN. Now with the WebPurify add-on, Cloudinary can automatically and quickly filter out problematic photos. This makes the decision of whether to moderate uploaded photos a no-brainer. You can focus on building great applications, allow your users to freely upload content to your app and let Cloudinary, together with WebPurify, make sure your app displays high quality images completely safe for any audience.

You are welcome to share any question or feedback you have regarding the automatic image moderation solution described in this post.

Recent Blog Posts

CoreMedia Adds Cloudinary to its CoreMedia Studio Platform

Today we’re pleased to announce a new technology partnership with CoreMedia, a leading Content Experience Platform provider. CoreMedia users can now leverage Cloudinary’s web-based digital asset management (DAM) solution to organize, search, manage and optimize their media assets, including images and videos, and to orchestrate, preview and deliver digital experiences consistently and optimized across all channels and browsers. The official press release is available here.

Read more
Facial-Surveillance System for Restricted Zones

In Africa, where Internet access and bandwidth are limited, it’s not cost-effective or feasible to establish and maintain a connectivity for security and surveillance applications. That challenge makes it almost impossible to build a service that detects, with facial-recognition technology, if someone entering a building is authorized to do so. To meet the final-year research requirement for my undergraduate studies, I developed a facial-surveillance system. Armed with a background in computer vision, I decided to push the limits and see if I could build a surveillance system that does not require recording long video footage.

Read more
Complex Networks Case Study

Complex Networks has been using Cloudinary since 2014 to manage and optimize images across seven websites and two mobile apps, making editorial workflow more efficient, improving page performance and load time, and increasing user engagement. Cloudinary was instrumental in enabling Complex Networks to redesign its web properties. Without the flexibility that Cloudinary offers to both creative and development teams, it would not have been possible for Complex Networks to achieve such a fast time to market.

Read more
Automate Placeholder Generation and Accelerate Page Loads

If you run a Google search on LQIP you’ll see very few relevant articles, very little guidance, and definitely no Wikipedia articles. In this post, we’ll discuss some of the feedback on LQIP we have gathered from the community and suggest and open for conversation a few approaches based on the built-in capabilities of the Cloudinary service. Specifically, we’ll explain what LQIP are, where they are best used, and how you can leverage them to accelerate page loads and optimize user experience.

Read more
Best Practices for Optimizing Web Page Speed

If you're like most consumers today, you engage more with pictures or videos on a website than text. The stats don't lie - four times as many visitors would rather watch a video about a product than read about it, and sites with compelling images average twice as many views as text-heavy ones.

Read more
A day of fun with Girls Who Code and Cloudinary

During both my computer science studies and work in the tech field, there have not been a lot of women present. While our ranks have grown, women still make up only a small percentage. In many ways, I think the traditionally male-dominated world can be intimidating to women and girls who may be interested in pursuing these types of tech careers.

Read more