How We Built the Generative AI Playground With Cloudinary: Web Development Meets Content Velocity

Six weeks, or a little over 1,000 hours (1,008 to be exact). That’s how long we gave ourselves to create a generative AI playground at ai.cloudinary.com.

Which begs the question, why the rush? 

First, we want to provide those interested with one place to confirm the power of our AI features ASAP. Second, Cloudinary has always been at the forefront of using AI for image classification, editing, and transformation; what better way to demonstrate this fact than to get them out into the wild quickly? And third, we wanted to underscore Cloudinary’s remarkable ability to expedite content creation. 

The result? A user-friendly, on-brand, user-generated content (UGC) demo site that transitioned from a concept to a fully operational platform. 

Let’s delve into how we pulled it off.

Every web development project starts with defining requirements. Although we knew Cloudinary’s tools and widgets could address some of our needs, we wanted a solid roadmap. Here’s what we established:

1. A single source of truth. We needed a central repository for all media used by the site: user uploads and site files.
2. Choose/upload an image. We aimed to offer users the flexibility to use predefined sample images or upload their own.
3. Moderate/normalize/categorize on upload. Uploaded content had to undergo normalization, moderation, and categorization.
4. Transform images on the fly. We wanted users to be able apply real-time transformations to images using our generative AI features, including fill, replace, remove, and recolor.
5. Tracking transformations. We needed a custom tracking system to monitor the transformations applied to the images.

It’s important to note that this list doesn’t encompass design and content requirements, which our Corporate Marketing team skillfully handled.

Outside of these requirements, we needed to decide on how to build the actual site. Selecting the proper framework sets the stage for the success of any web development project. We chose Next.js 13.

Here’s why:

1. Flexible and powerful. Next.js is a robust next-generation JavaScript framework built on the sturdy base of React.
2. Seamless integration. Cloudinary easily integrates with React and Next.js, streamlining our development process.
3. Ease of deployment. Next.js effortlessly deploys to renowned hosting solutions like Netlify and Vercel, ensuring a smooth deployment process.

Establishing a centralized hub for our media assets was an effortless task, thanks to Cloudinary’s Programmable Media and Assets. Using these tools we devised a custom taxonomy tailored to our site’s file organization. Storing sample images was as simple as using Cloudinary Assets. We organized dedicated folders for each demo to keep the uploads and sample images separate. 

The uploading of images was streamlined by integrating our upload widget alongside Director of Developer Experience Engineering Colby Fayock’s Next.js/Cloudinary plugin. Colby’s plugin simplified the integration of the robust upload widget with Next.js websites, requiring minimal coding, and offering prebuilt components.

In an environment centered on UGC, Cloudinary provided various tools to streamline content management.

1. Normalization. Cloudinary offered a straightforward solution for normalizing uploaded content. We used incoming transformations to crop and scale incoming images to ensure a unified experience and faster loading times.
2. Moderation. We seamlessly implemented AWS Rekognition content moderation to ensure all uploads didn’t violate our terms and conditions and did not include anything inappropriate.
3. Categorization. Cloudinary’s tools facilitated the efficient categorization of content, ensuring a well-organized repository. AWS Rekognition auto-tagging came in handy here.

All the transformations displayed in our demos were generated through Cloudinary’s Programmable Media. When users entered their prompts in the user interface, the site’s logic translated the input into parameters that could be used with Cloudinary. Programmable Media’s transformation API delivered these images upon request. The resulting URLs were also presented to users, providing a real-time display of the transformations.

We track web traffic with Google Analytics, but we also wanted to track some custom information, such as daily transformation counts broken out by feature and moderation status. To achieve this, we created a custom integration with Zapier, which writes all the transformation data to a custom datastore in Google Cloud. We also send a record of each to a custom channel on our internal Slack. Zapier is an excellent tool for automating these features, so naturally, Cloudinary has a Zapier integration that’s worth checking out.

In under six weeks, Cloudinary played a pivotal role in helping us construct this demo site, a timeline encompassing wireframing, design, and rigorous testing. We swiftly brought this site to life by leveraging our great tools, including the Cloudy/Next plugin and our efficient upload widget. Within a short span, we’re witnessing hundreds of transformations submitted daily by users. We invite you to explore our Generative AI at ai.cloudinary.com.