Build a YouTube Shorts Creator With Highlighted Subtitles in Next.js

YouTube Shorts are a powerful way to reach a wider audience with short, engaging videos. However, many creators face a common challenge: transforming widescreen videos into the vertical format required for YouTube Shorts without losing the key parts of the frame.

This blog post explains how to automatically transform wide videos into YouTube Shorts using Cloudinary’s smart cropping feature and how to add auto-generated, highlighted subtitles with Google Speech-to-Text to make your content more engaging and accessible.

Here’s a demo preview of what you’ll be building:

Before getting into the setup, you can check out the full code on GitHub.

To get started, create a new Next.js project by running the following command:

npx create-next-app@14 youtube-shorts-creator

cd youtube-shorts-creatorCode language: CSS (css)

For this blog post, you’ll use Next.js 14 and server actions to handle video uploads (Next.js 13 can also be used as it supports Sever Actions). Once your project is set up, install the Cloudinary SDK with the command below:

<code>npm install cloudinary</code>Code language: HTML, XML (xml)

You’ll need a Cloudinary account to access your credentials, which allow you to interact with the Cloudinary SDK. If you don’t have an account yet, sign up at Cloudinary.

After signing in, go to your Cloudinary dashboard and copy the following details: Cloud Name, API Key, and API Secret.

Now, in your Next.js project, create a .env.local file at the root and add your Cloudinary credentials like this:

NEXT_PUBLIC_CLOUDINARY_CLOUD_NAME=your_cloud_name

NEXT_PUBLIC_CLOUDINARY_API_KEY=your_api_key

CLOUDINARY_API_SECRET=your_api_secret

Replace your_cloud_name, your_api_key, and your_api_secret with your actual Cloudinary credentials from the dashboard.

With the Cloudinary SDK installed and credentials set up, let’s implement the logic to upload videos from the frontend and process them through Cloudinary. 

You’ll start by creating a form to allow users to upload their videos, then handle these uploads using server actions.

Create a file upload form in your page.tsx. This form will allow users to select a video from their local device and submit it for upload to Cloudinary.

Here’s a basic form setup. You can paste this into your page.tsx file:

'use client';

import { useState } from 'react';

export default function Home() {

    const[loading, setLoading]= useState(false);

    const handleSubmit = async (event: React.FormEvent<HTMLFormElement>)=>{

        event.preventDefault();

        setLoading(true);

        // Get the file from the form

const formData =new FormData(event.currentTarget);

        try {

            // Upload the video to Cloudinary

} catch (error){

            console.error('Upload failed:', error);

} finally {

            setLoading(false);

}

};

    return (

        <div>

            <h1>Upload a Video</h1>

            <form onSubmit={handleSubmit}>

                <input type="file" name="video" accept="video/*" required />

                <button type="submit" disabled={loading}>

                    {loading ? 'Uploading...' : 'Upload Video'}

                </button>

            </form>

        </div>

    );

}Code language: JavaScript (javascript)

In the code above, the form allows users to upload a video. When the form is submitted:

• The handleSubmit function extracts the video file using FormData.
• The loading state ensures the button is disabled during the upload, displaying “Uploading…” while the process runs.
• A try-catch block handles any errors that may occur during the upload.

The actual upload logic will be added after the server action for uploading the video to Cloudinary is handled.

In this step, let’s handle the video upload on the server side with server actions. This ensures that sensitive operations, like handling API keys, are kept on the server side.

Create an actions/upload.ts file and configure Cloudinary to handle the video upload:

'use server';

import { v2 as cloudinary } from 'cloudinary';

cloudinary.config({

  cloud_name: process.env.NEXT_PUBLIC_CLOUDINARY_CLOUD_NAME,

  api_key: process.env.NEXT_PUBLIC_CLOUDINARY_API_KEY,

  api_secret: process.env.CLOUDINARY_API_SECRET,

});Code language: JavaScript (javascript)

Once you have configured Cloudinary, set up an upload function to handle video uploads to Cloudinary:

export async function upload(formData: FormData) {

    const file = formData.get('video') as File;

    const buffer: Buffer = Buffer.from(await file.arrayBuffer());

    // Sanitize the public_id (file name) if needed

    const safePublicId = file.name.replace(/[^a-zA-Z0-9-_]/g,'_');

    const uploadResponse = await new Promise<{

        secure_url: string;

        public_id: string;

}>((resolve, reject)=>{

        cloudinary.uploader

            .upload_stream(

{

                    resource_type:'video',

                    public_id: safePublicId,

},

(error, result)=>{

if(error){

                        reject(error);

}elseif(result){

                        resolve(result);

}else{

                        reject(new Error('Upload result is undefined'));

}

}

)

            .end(buffer);

});

    // Return the original URL and the video ID

    return {

        originalUrl: uploadResponse.secure_url,

        videoId: uploadResponse.public_id,

    };

}Code language: JavaScript (javascript)

In the code above, the video file is converted to a buffer file before being uploaded to Cloudinary’s upload_stream function. Once the video is uploaded, the response includes important details, such as the video URL (secure_url) and the video identifier (public_id).

The originalUrl and videoId are returned to the frontend. The videoId will be critical when you implement transformations on the client side, as Cloudinary uses this ID to reference and modify the video (e.g., applying subtitles or cropping).

Modify the frontend to connect it with the server-side upload function. This will allow the form submission to trigger the video upload process.

Here’s how you update your page.tsx to integrate the server-side upload logic:

// Import the upload function

import { upload } from '../actions/upload';

const handleSubmit = async (event: React.FormEvent<HTMLFormElement>) => {

    event.preventDefault();

    // Show loading state

    setLoading(true);

    const formData = new FormData(event.currentTarget);

    try {

        // Call the upload function to upload the video

        const result = await upload(file); 

        // Log the video URL & video ID

        console.log('Uploaded video URL:', result.originalUrl);

        console.log('Uploaded video ID:', result.videoId); 

    } catch (error) {

        // Handle any errors that occur during upload

        console.error('Upload failed:', error);

    } finally {

        // End loading state

        setLoading(false);

    }

};Code language: JavaScript (javascript)

In the code above, the upload function is called. When the form is submitted, the selected video file is passed to the server action, which uploads it to Cloudinary. 

The server then returns the originalUrl (to display the video) and the videoId (for future transformations like adding subtitles or cropping). These values are logged to the console for now, but they’ll be used in the next steps.

Once the video is uploaded, it will be displayed using the originalUrl that Cloudinary provides. Update the page.tsx file to store the video URL and display it in a video player:

const [videoUrl, setVideoUrl] = useState<string | null>(null); // Store video URL

const[videoId, setVideoId]= useState<string | null>(null);

const handleSubmit = async (event: React.FormEvent<HTMLFormElement>) => {

    event.preventDefault();

    setLoading(true);

    const formData = new FormData(event.currentTarget);

    try {

        const result = await upload(formData); // Upload the video

        setVideoUrl(result.originalUrl); // Set the video URL in state

        setVideoId(result.videoId);// Set the video ID for further transformations

    } catch (error) {

        console.error('Upload failed:', error);

    } finally {

        setLoading(false);

    }

};

// Render the uploaded video

return (

    <div>

        <h1>Upload a Video</h1>

        <form onSubmit={handleSubmit}>

            <input type="file" name="video" accept="video/*" required />

            <button type="submit" disabled={loading}>

                {loading ? 'Uploading...' : 'Upload Video'}

            </button>

        </form>

        {videoUrl && (

            <div>

                <h2>Uploaded Video:</h2>

                <video src={videoUrl} controls className="w-full max-w-lg" />

            </div>

        )}

    </div>

);Code language: JavaScript (javascript)

In this step, the videoUrl is stored in the component’s state using setVideoUrl. Once the video is successfully uploaded, the URL is rendered inside an HTML <video> element, allowing the user to watch the uploaded video directly on the page.

Now, users can select a video, upload it to Cloudinary, and see the uploaded video displayed on the screen. Store the videoIdin a state variable, as you’ll use it in the next section to transform the video by adding subtitles and cropping it to the right format.

After you upload your video to Cloudinary, you can adjust it using Cloudinary’s smart cropping feature, which allows you to adjust wide videos into a vertical format for YouTube Shorts. 

This feature analyzes the content of the video and adjusts the frame to focus on the main subject or action. In this case, let’s focus on cropping the video to a vertical format using the c_fill, w, and h transformation parameters:

• c_fill. Crops the video to ensure the output fills the specified dimensions.
• w and h. These define the width and height of the video. In this case, let’s set the width to 720 and the height to 1280 to create a vertical video. You can modify this dimension to fit whatever platform specification you desire.

Here’s an example transformation URL:

https://res.cloudinary.com/${cloudName}/video/upload/c_fill,w_720,h_1280/${videoId}.mp4Code language: JavaScript (javascript)

This URL takes the uploaded video and transforms it to the specified width and height, filling the frame while ensuring the most important part of the video remains in focus.

Now that you know what the transformation URL will look like, you can display the original video uploaded to Cloudianry and the transformed video side by side. 

Here’s how to render both videos in the page.tsx file:

{videoUrl && transformedVideoUrl && (

    <div className="flex justify-center space-x-4 mt-10">

        {/* Original Video */}

        <div>

            <h2 className="text-lg font-bold text-center mb-4">Uploaded Video:</h2>

            <video src={videoUrl} controls className="w-full max-w-md border-4 rounded" />

        </div>

        {/* Transformed Video */}

        <div>

            <h2 className="text-lg font-bold text-center mb-4">Transformed Video (Smart Cropping):</h2>

            <video 

                src={`https://res.cloudinary.com/${process.env.NEXT_PUBLIC_CLOUDINARY_CLOUD_NAME}/video/upload/c_fill,w_720,h_1280/${videoId}.mp4`} 

                controls 

                className="w-full max-w-md border-4 rounded" 

            />

        </div>

    </div>

)}Code language: HTML, XML (xml)

The code above assigns the transformed URL, which incorporates Cloudinary’s transformation parameters, to the src attribute of the second <video> tag. This ensures the tag dynamically loads the transformed video, allowing you to display the smart-cropped version alongside the original.

Cloudinary integrates with Google’s Speech-to-Text to auto-generate subtitles based on the audio in the uploaded video. Once generated, these subtitles can be applied as an overlay on the video.

To auto-generate subtitles using Google Speech-to-Text, modify the upload process by adding a raw_convert transformation. This tells Cloudinary to send the video to Google’s Speech-to-Text API and return a subtitle file in the form of .transcript.

Before you do this, ensure you search for the Google AI Video Transcription add-on on your Cloudinary dashboard:

Click the add-on and subscribe to the free plan:

Once Google AI Video Transcription is set up, you can make API calls to Cloudinary. Modify the uploadResponse upload server function to include subtitle generation:

const uploadResponse = await new Promise<{

    secure_url: string;

    public_id: string;

}>((resolve, reject)=>{

    cloudinary.uploader

        .upload_stream(

{

                resource_type:'video',

                public_id: safePublicId,

                raw_convert:'google_speech',// Request transcription

},

(error, result)=>{

if(error){

                    reject(`Upload failed: ${error.message}`);

}else{

                    resolve(result);

}

}

)

        .end(buffer);

});Code language: JavaScript (javascript)

Once the subtitles are generated, you can overlay them on the transformed video using Cloudinary’s l_subtitles transformation or by adding the transcript file as a track to the video. For this demo, we’ll use the latter option.

To include the transcript as a separate track for a video player, you can request Cloudinary to generate an SRT and/or WebVTT file by using the google_speech value with the srt or vtt qualifiers. For example, to upload a video and request a WebVTT (.vtt) file for the transcript, set the raw_convert parameter as follows during the upload process:

raw_convert:'google_speech:vtt',

With the generated transcript file, you can now modify the transformed video’s <video> tag to include both the video source and the subtitles track using the <track> tag:

<video

  crossOrigin="anonymous"

  controls

  className="w-full max-w-md border-4 rounded"

>

  <source

    id="mp4"

    src={`https://res.cloudinary.com/${process.env.NEXT_PUBLIC_CLOUDINARY_CLOUD_NAME}/video/upload/c_fill,w_720,h_1280/${videoId}.mp4`}

    type="video/mp4"

  />

  <track

    label="English"

    kind="subtitles"

    srcLang="en"

    src={`https://res.cloudinary.com/${process.env.NEXT_PUBLIC_CLOUDINARY_CLOUD_NAME}/raw/upload/${videoId}.vtt`}

    default

  />

</video>Code language: HTML, XML (xml)

In this setup:

• The <source> tag specifies the transformed video file.
• The <track> tag references the WebVTT file containing the subtitles, allowing them to be displayed in the video player.

This method ensures subtitles are accessible and can be toggled by users. For more details on formatting subtitles, refer to Cloudinary’s documentation.

This blog post demonstrates how to transform wide videos into vertical YouTube Shorts with smart cropping, and auto-generated subtitles, all using Cloudinary and Next.js. Although we focused on creating YouTube Shorts, the same techniques can be easily applied to videos for platforms like TikTok, Instagram Reels, and Facebook.

By combining Cloudinary’s powerful video transformation capabilities with a simple upload interface, creators can easily convert their videos into the required format for social media platforms while adding professional features like subtitles. Sign up for a free account today to try it for yourself.

If you enjoyed this post and want to discuss it more, join the Cloudinary Community forum and its associated Discord.

• Cloudinary Video Documentation
• Cloudinary Overall Media Documentation
• Google AI Video Transcription
• Video transformations
• Next.js Documentation