Cloudinary Blog

Migrating your Media Assets to the Cloud Using Cloudinary

Easily Migrate your Media Assets to the Cloud with Cloudinary

When analyzing use of website or mobile application storage, there’s no doubt that media files, such as video clips and images, use the most space. Migrating these files to the cloud and storing them in a location where they are available online makes perfect sense, but images and videos often require additional transformations before they are delivered to end users.

Images need to be transformed to optimal formats, resized and cropped for various aspect ratios, especially if you have a responsive site. In addition, you may want to apply special effects and overlays. Videos, too, need to be optimized for web delivery, transcoded to support different resolution options and streamed using adaptive streaming, as well as making other modifications. Then, when your media files are stored on the cloud, you’ll want fast delivery via content deliver networks (CDNs) to ensure a great user experience.

Cloudinary, a cloud-based service that supports all image and video management needs for websites and mobile apps, delivers all of these capabilities. In order to start using Cloudinary, you first need to set up a free plan to start. Then you will have to migrate your images and videos to your Cloudinary account. Here’s how you get started:

Requirements

The code samples provided here are in Node.js. However Cloudinary publishes SDKs in several languages. You can find the various options listed here.

In case your development language isn’t there, Cloudinary supports the option of uploading images using a direct HTTP call to the API, as explained here.

Image manipulation and delivery also can be implemented by providing the transformation parameters as part of the URL directly.

Installation

  1. Open a Cloudinary free account.
    Go to Cloudinary and click “sign up for free.” Fill out the registration form. Here you can change your default cloud name to your preferred name. In these examples my cloud name is “cld-name.”

  2. Follow the instructions according to the required SDK documentation as listed here.

Migration Planning

Much like when you move to a new apartment, you need to ask yourself, do I need to move all my files or is it a good opportunity to leave some of it behind? To answer those questions, let’s consider several migration options, each fitting with a different scenario.

Migrating all existing content in one phase – This requires uploading all your media files. This option fits one of the following use cases:

  • All of your images and videos are actively used.
  • You intend to shut down your existing media storage.

Implement this migration by creating a script that runs on your media assets and for each file it calls the upload API call. Cloudinary’s upload API call supports uploading files from various sources, including a local path, a remote HTTP or HTTPS URL, a remote S3 URL, a Data URI or an FTP URL.

When uploading a file, you can define your own unique Id (as shown below). If it is not defined, the call will create one randomly. Here is an example of uploading a file from your local computer:

cloudinary.v2.uploader.upload("local_folder/image.jpg", {public_id: "my_image"}, 
function(error, result) { console.log(error,result) });

Lazy migration – Uploading a media file only when it is requested by your website or app user for the first time. This option is effective when you have a long tail of media assets, not all of them are active, and you are not sure which ones are still in use.

Using the Cloudinary management console or the API, you can define a mapping between a folder name in your Cloudinary media library and a base remote URL, which links to your images online folder. For example, if your images are available at: https://example.fileserver.com/media/, the API call shown below will map https://example.fileserver.com/media/ to a Cloudinary folder called: media

cloudinary.v2.api.create_upload_mapping('media',
    { template: "https://example.fileserver.com/media/" },
    function(error, result) { console.log(error,result) });

Now, in order to automatically upload koala.jpg, you need to replace, on your front end, the image URL:

https://example.fileserver.com/media/koala.jpg

With the following URL:

https://res.cloudinary.com/cld-name/image/upload/media/koala.jpg

The first user who calls the Cloudinary URL will trigger an automatic upload of the image to Cloudinary. Any subsequent request for the same image will be delivered via the CDN.

The Cloudinary URL is constructed as follows:

res.cloudinary.com/<cloud-name>/image/upload/<mapped-folder>/<partial-path-of-remote-image>*

The following API call also will return the required URL:

cloudinary.url("media/koala.jpg");

Hybrid approach – Run a script to upload the “hot” group of your most commonly used media assets and use the “lazy migration” option to upload the rest. This option works best when you have a defined subset of your media assets that drives most of your traffic.

Fetch assets – Fetch media assets from remote locations and store them for a predefined period. Use this option when your images and videos originate from various online sources and they are used for a short timespan, as in news items.

For example, the following code is used to deliver a remote image of Jennifer Lawrence fetched by Cloudinary from WikiMedia.

cloudinary.url("http://upload.wikimedia.org/wikipedia/commons/4/46/Jennifer_Lawrence_at_the_83rd_Academy_Awards.jpg", {type: "fetch"});

The equivalent URL is:

http://res.cloudinary.com/cld-name/image/fetch/http://upload.wikimedia.org/wikipedia/commons/4/46/Jennifer_Lawrence_at_the_83rd_Academy_Awards.jpg

Uploading Large Files – If you are uploading files larger than 100MB, there is an option to do a chunked upload:

cloudinary.v2.uploader.upload_large("my_large_image.tiff", 
    { resource_type: "image", chunk_size: 6000000 }, 
    function(error, result) { console.log(error,result) });

Which files to upload?

Cloudinary is able to manipulate images and videos on-the-fly upon request or upon upload, so you only need to upload the highest resolution of one image or video. There is no need to upload large/medium/small variants.

Access Control

If all of your media assets are not public, you can upload them and restrict their availability:

  • Private files – The original files are not available unless accessed via a signed URL. Another option to provide access is creating a time-expired download URL. The following code example uploads the file as private:
cloudinary.v2.uploader.upload("local_folder/private_image.jpg", {type: "private"}, 
function(error, result) { console.log(error,result) });
  • Authenticated files - The original files, as well as their derived ones, are not available unless accessed via a signed URL. For increased security, cookie-based authentication can be setup as well, in order to access them.

  • Whitelisting referral domains – An additional optional security layer that restricts the access to your media assets is to setup a whitelist of referral domains. Then only URL requests arriving from those domains are allowed to access the file.

Setting an Upload Policy Using an Upload Preset

A convenient way to create a centralized upload policy is defining an upload preset. This enables you to define the transformations you would like to do once, then use the preset name to activate it upon upload. You can define several upload presets and use them according to different policies you have, for example watermark all images or transcode a video rendition of 640p wide resolution.

When you define an upload preset, you can set a transformation that will change the original file and then only the transformed file will be stored. This option is called an incoming transformation. You can also define transformations that will be created as derived files, which will be stored in addition to the original file. This process is called an eager transformation. Using Cloudinary, you can transform the images and video on-the-fly, therefore these options are required for cases where you would like to process the transformation immediately upon upload.

As an example, the following code creates an upload preset that adds the tag remote. The unsigned parameter determines if the preset can be used for unsigned uploads, which can be done from the client side without having the API secret. The allowed_formats parameter defines the file formats allowed to be used with this preset.

cloudinary.v2.api.create_upload_preset({ name: "my_preset", 
    unsigned: true, tags: "remote", allowed_formats: "jpg,png" },
    function(error, result) { console.log(error,result) });

The following code uploads an image using this upload preset:

cloudinary.v2.uploader.upload("smiling_man.jpg", 
    { public_id: “smile”, upload_preset: "my_preset" }, 
    function(error, result) { console.log(error,result) });

Upload Response

As a response to the upload call, you get some important information back. The response looks like this:

{ public_id: 'smile',
  version: 1482935950,
  signature: 'bfd5019ee4f513f30226cc06c750b2ad6eccceef',
  width: 1743,
  height: 1307,
  format: 'jpg',
  resource_type: 'image',
  created_at: '2016-12-28T14:39:10Z',
  tags: [],
  bytes: 642781,
  type: 'upload',
  etag: '7c26a0d7b72b6621bba1110da25d099e',
  url: 'http://res.cloudinary.com/hadar-staging/image/upload/v1482935950/smile.jpg',
  secure_url: 'https://res.cloudinary.com/hadar-staging/image/upload/v1482935950/smile.jpg',
  original_filename: 'smiling_man' }

Notification URL

You can tell Cloudinary to notify your application as soon as the upload completes by adding the notification_url parameter to the upload method and setting it to any valid HTTP or HTTPS URL. You also can set the notification_url parameter globally for all uploads on the Upload Settings page in the Cloudinary Management Console, instead of individually for each upload call.

Uploading New Files

Following a successful migration, you need to start uploading all new media assets to Cloudinary. There are several ways to do it: manually via the Cloudinary account console, calling the upload API, or using automatic upload. Another easy way to do this is using the ready-made upload widget.

Upload Widget

Cloudinary's upload widget includes a complete graphical interface. The widget supports a drag and drop functionality, interactive cropping, upload progress indication and thumbnail previews. The widget also monitors and handles uploading errors. The following code example shows how to open the widget:

<script src="//widget.cloudinary.com/global/all.js" type="text/javascript">  
</script>
<script>
  cloudinary.setCloudName('cld-name');
  cloudinary.openUploadWidget({upload_preset: 'my_preset'}, 
  function(error, result) {console.log(error, result)});
</script>

More information regarding the widget functionality is available here.

Summary

The steps detailed above are just the beginning of the journey of moving your media assets to the cloud. Once uploaded, Cloudinary supports a long list of image manipulation options, and the same goes for video. In addition, images can be optimized for faster delivery and support responsive design.

Now is the time to let go, send your media assets to the cloud and set them free.

Recent Blog Posts

Embed Images in Email Campaigns at Scale

By Sourav Kundu
Embed Images in Email Campaigns at Scale

We live in an era of information overload and attention is the real currency. Marketers are constantly looking for new ways to reach you, to advertise products and services that they think could improve your lives.

Read more
Build the Back-End For Your Own Instagram-style App with Cloudinary

Github Repo

Managing media files (processing, storage and manipulation) is one of the biggest challenges we encounter as practical developers. These challenges include:

A great service called Cloudinary can help us overcome many of these challenges. Together with Cloudinary, let's work on solutions to these challenges and hopefully have a simpler mental model towards media management.

Read more

Build A Miniflix in 10 Minutes

By Prosper Otemuyiwa
Build A Miniflix in 10 Minutes

Developers are constantly faced with challenges of building complex products every single day. And there are constraints on the time needed to build out the features of these products.

Engineering and Product managers want to beat deadlines for projects daily. CEOs want to roll out new products as fast as possible. Entrepreneurs need their MVPs like yesterday. With this in mind, what should developers do?

Read more

Your Web Image is Unnecessarily Bloated

By Christian Nwamba
Your Web Image is Unnecessarily Bloated

As a developer, it seems inefficient to serve a 2000kb JPEG image when we could compress images to optimize the performance without degrading the visual quality.

We are not new to this kind of responsibility. But our productivity will end up being questioned if we do not deliver fast. In order to do so, the community has devised several patterns to help improve productivity. Let's review few of these patterns based on their categories:

Read more

Google For Nigeria: We saw it all…

By Christian Nwamba
Google For Nigeria: We saw it all…

Note from Cloudinary: Christian Nwamba, a frequent Cloudinary contributor, recently attended, and was a main speaker, at the Google Developer Group (GDG) Conference in Lagos, Nigeria. Christian led a session teaching more than 500 developers how to “Build Offline Apps for the Next Billion Users.” The stack he used included JS (Vue), Firebase, Service Workers and Cloudinary. Below is his account of the conference and his talk.

Read more
Viral Images: Securing Images and Video uploads to your systems

When was the last time you got paid $40,000 for a few days of work? That is what happened last year to Russian independent security researcher Andrey Leonov, who discovered that if you upload a specially constructed image file to Facebook, you can make Facebook's internal servers, nested deep within their firewalls, run arbitrary commands to expose sensitive internal files in a way that could easily lead to a data breach.

Read more