Ask AI
AI Video Analysis API (Beta)
Last updated: Jul-22-2026
Overview
The AI Video Analysis API uses AI to generate a visual transcription of a video: a sequence of timestamped, natural-language descriptions of what's visible on screen. Unlike video transcription, which converts spoken audio to text, a visual transcription describes the visual content itself, such as scenes, subjects, actions, and settings, even when the video has no audio.
You submit a video asset that's already stored in your product environment, optionally guide the analysis with a prompt, and receive a transcript file containing one entry per scene segment.
Common use cases include:
- Accessibility: Use the descriptions as the basis for an audio description (AD) script that helps people who are blind or have low vision understand the visual content. See using the output for accessibility.
- Visual search and tagging: Feed the descriptions into a MediaFlows workflow or an LLM to map visual content to your own tags or taxonomy.
- Content understanding: Generate scene-by-scene summaries of video content for editorial or cataloging workflows.
How it works
Analysis runs asynchronously:
- Send a
POSTrequest with the asset to analyze. The response returns ajob_idand an initial status ofpending. - Poll the job using the
job_iduntil the status iscompleted. - The completed response includes a
visual_transcriptionobject that points to a raw file containing the timestamped descriptions.
Generate a visual transcription
Send a POST request to the ai_video_analysis endpoint:
https://api.cloudinary.com/v2/video/<cloud_name>/ai_video_analysis
Authenticate using your API key and secret. Include the following parameters in your JSON request body:
| Parameter | Type | Required | Description |
|---|---|---|---|
video_asset_id |
String | Yes | The unique identifier of the video asset to analyze. The video must already be stored in your product environment. You can retrieve an asset's ID from the upload response or by using the Admin API. |
visual_transcription_prompt |
String | No | A natural-language instruction that guides the description style or focus, for example "focus on describing colors". If omitted, a default prompt is used. |
Example request
Example response
The request returns a 201 response with a job_id and a status of pending:
Check the analysis status
To retrieve the result, send a GET request to the same endpoint with the job_id appended to the path:
https://api.cloudinary.com/v2/video/<cloud_name>/ai_video_analysis/<job_id>
Example request
While the analysis is in progress, the status is pending. Once it finishes, the status is completed and the response includes a visual_transcription object:
The visual_transcription object describes the generated transcript file, which is stored as a raw asset in your product environment:
| Field | Description |
|---|---|
asset_id |
The unique identifier of the generated transcript file. |
public_id |
The public ID of the transcript file. |
resource_type |
The resource type of the file. Always raw. |
delivery_type |
The delivery type of the file, for example upload. |
url |
The URL to fetch the transcript file contents. |
The visual transcription file
Fetch the file at the returned url to retrieve the descriptions. The file contains a JSON array of segments, where each segment describes a portion of the video over a time range (in seconds).
For example, analyzing this video:
produces the following transcript:
| Field | Description |
|---|---|
transcript |
A natural-language description of what's visible during the segment. |
start_time |
The start of the segment, in seconds. |
end_time |
The end of the segment, in seconds. |
Guiding the output with a prompt
By default, the API produces a general description of each segment. Use the optional visual_transcription_prompt parameter to steer what the descriptions focus on or how they're phrased.
For example, analyzing the same video with the prompt "focus on describing colors":
produces color-rich descriptions:
You might instead focus the model on actions, settings, or on-screen objects, depending on your use case.
Using the output for accessibility
A visual transcription provides the raw material for an audio description (AD) script, which conveys important visual information to people who are blind or have low vision. Because each segment is timestamped, you can present the descriptions as a time-aligned text track or as text alongside the video for screen readers.
For example, the color-focused turtle transcription above maps directly onto a WebVTT file, with each segment's start_time and end_time becoming a cue:
Upload this file to your product environment and supply it as a descriptions text track in the Cloudinary Video Player, as shown in Audio descriptions as captions. Review and edit the generated text before publishing, so that the descriptions are accurate and fit within the available gaps in any existing audio.
Error handling
If a request can't be processed, the response includes an error object with a message. For example, an invalid asset identifier returns:
Beta limitations
During Beta, the API returns the visual transcription file only. Additional outputs, such as a summary, title, or tags derived from the analysis, are planned for a future phase.
Related topics
Error
Rate this page:
