Cloudinary Blog

Migrating your Media Assets to the Cloud Using Cloudinary

Easily Migrate your Media Assets to the Cloud with Cloudinary

When analyzing use of website or mobile application storage, there’s no doubt that media files, such as video clips and images, use the most space. Migrating these files to the cloud and storing them in a location where they are available online makes perfect sense, but images and videos often require additional transformations before they are delivered to end users.

Images need to be transformed to optimal formats, resized and cropped for various aspect ratios, especially if you have a responsive site. In addition, you may want to apply special effects and overlays. Videos, too, need to be optimized for web delivery, transcoded to support different resolution options and streamed using adaptive streaming, as well as making other modifications. Then, when your media files are stored on the cloud, you’ll want fast delivery via content deliver networks (CDNs) to ensure a great user experience.

Cloudinary, a cloud-based service that supports all image and video management needs for websites and mobile apps, delivers all of these capabilities. In order to start using Cloudinary, you first need to set up a free plan to start. Then you will have to migrate your images and videos to your Cloudinary account. Here’s how you get started:

Requirements

The code samples provided here are in Node.js. However Cloudinary publishes SDKs in several languages. You can find the various options listed here.

In case your development language isn’t there, Cloudinary supports the option of uploading images using a direct HTTP call to the API, as explained here.

Image manipulation and delivery also can be implemented by providing the transformation parameters as part of the URL directly.

Installation

  1. Open a Cloudinary free account.
    Go to Cloudinary and click “sign up for free.” Fill out the registration form. Here you can change your default cloud name to your preferred name. In these examples my cloud name is “cld-name.”

  2. Follow the instructions according to the required SDK documentation as listed here.

Migration Planning

Much like when you move to a new apartment, you need to ask yourself, do I need to move all my files or is it a good opportunity to leave some of it behind? To answer those questions, let’s consider several migration options, each fitting with a different scenario.

Migrating all existing content in one phase – This requires uploading all your media files. This option fits one of the following use cases:

  • All of your images and videos are actively used.
  • You intend to shut down your existing media storage.

Implement this migration by creating a script that runs on your media assets and for each file it calls the upload API call. Cloudinary’s upload API call supports uploading files from various sources, including a local path, a remote HTTP or HTTPS URL, a remote S3 URL, a Data URI or an FTP URL.

When uploading a file, you can define your own unique Id (as shown below). If it is not defined, the call will create one randomly. Here is an example of uploading a file from your local computer:

cloudinary.v2.uploader.upload("local_folder/image.jpg", {public_id: "my_image"}, 
function(error, result) { console.log(error,result) });

Lazy migration – Uploading a media file only when it is requested by your website or app user for the first time. This option is effective when you have a long tail of media assets, not all of them are active, and you are not sure which ones are still in use.

Using the Cloudinary management console or the API, you can define a mapping between a folder name in your Cloudinary media library and a base remote URL, which links to your images online folder. For example, if your images are available at: https://example.fileserver.com/media/, the API call shown below will map https://example.fileserver.com/media/ to a Cloudinary folder called: media

cloudinary.v2.api.create_upload_mapping('media',
    { template: "https://example.fileserver.com/media/" },
    function(error, result) { console.log(error,result) });

Now, in order to automatically upload koala.jpg, you need to replace, on your front end, the image URL:

https://example.fileserver.com/media/koala.jpg

With the following URL:

https://res.cloudinary.com/cld-name/image/upload/media/koala.jpg

The first user who calls the Cloudinary URL will trigger an automatic upload of the image to Cloudinary. Any subsequent request for the same image will be delivered via the CDN.

The Cloudinary URL is constructed as follows:

res.cloudinary.com/<cloud-name>/image/upload/<mapped-folder>/<partial-path-of-remote-image>*

The following API call also will return the required URL:

cloudinary.url("media/koala.jpg");

Hybrid approach – Run a script to upload the “hot” group of your most commonly used media assets and use the “lazy migration” option to upload the rest. This option works best when you have a defined subset of your media assets that drives most of your traffic.

Fetch assets – Fetch media assets from remote locations and store them for a predefined period. Use this option when your images and videos originate from various online sources and they are used for a short timespan, as in news items.

For example, the following code is used to deliver a remote image of Jennifer Lawrence fetched by Cloudinary from WikiMedia.

cloudinary.url("http://upload.wikimedia.org/wikipedia/commons/4/46/Jennifer_Lawrence_at_the_83rd_Academy_Awards.jpg", {type: "fetch"});

The equivalent URL is:

http://res.cloudinary.com/cld-name/image/fetch/http://upload.wikimedia.org/wikipedia/commons/4/46/Jennifer_Lawrence_at_the_83rd_Academy_Awards.jpg

Uploading Large Files – If you are uploading files larger than 100MB, there is an option to do a chunked upload:

cloudinary.v2.uploader.upload_large("my_large_image.tiff", 
    { resource_type: "image", chunk_size: 6000000 }, 
    function(error, result) { console.log(error,result) });

Which files to upload?

Cloudinary is able to manipulate images and videos on-the-fly upon request or upon upload, so you only need to upload the highest resolution of one image or video. There is no need to upload large/medium/small variants.

Access Control

If all of your media assets are not public, you can upload them and restrict their availability:

  • Private files – The original files are not available unless accessed via a signed URL. Another option to provide access is creating a time-expired download URL. The following code example uploads the file as private:
cloudinary.v2.uploader.upload("local_folder/private_image.jpg", {type: "private"}, 
function(error, result) { console.log(error,result) });
  • Authenticated files - The original files, as well as their derived ones, are not available unless accessed via a signed URL. For increased security, cookie-based authentication can be setup as well, in order to access them.

  • Whitelisting referral domains – An additional optional security layer that restricts the access to your media assets is to setup a whitelist of referral domains. Then only URL requests arriving from those domains are allowed to access the file.

Setting an Upload Policy Using an Upload Preset

A convenient way to create a centralized upload policy is defining an upload preset. This enables you to define the transformations you would like to do once, then use the preset name to activate it upon upload. You can define several upload presets and use them according to different policies you have, for example watermark all images or transcode a video rendition of 640p wide resolution.

When you define an upload preset, you can set a transformation that will change the original file and then only the transformed file will be stored. This option is called an incoming transformation. You can also define transformations that will be created as derived files, which will be stored in addition to the original file. This process is called an eager transformation. Using Cloudinary, you can transform the images and video on-the-fly, therefore these options are required for cases where you would like to process the transformation immediately upon upload.

As an example, the following code creates an upload preset that adds the tag remote. The unsigned parameter determines if the preset can be used for unsigned uploads, which can be done from the client side without having the API secret. The allowed_formats parameter defines the file formats allowed to be used with this preset.

cloudinary.v2.api.create_upload_preset({ name: "my_preset", 
    unsigned: true, tags: "remote", allowed_formats: "jpg,png" },
    function(error, result) { console.log(error,result) });

The following code uploads an image using this upload preset:

cloudinary.v2.uploader.upload("smiling_man.jpg", 
    { public_id: “smile”, upload_preset: "my_preset" }, 
    function(error, result) { console.log(error,result) });

Upload Response

As a response to the upload call, you get some important information back. The response looks like this:

{ public_id: 'smile',
  version: 1482935950,
  signature: 'bfd5019ee4f513f30226cc06c750b2ad6eccceef',
  width: 1743,
  height: 1307,
  format: 'jpg',
  resource_type: 'image',
  created_at: '2016-12-28T14:39:10Z',
  tags: [],
  bytes: 642781,
  type: 'upload',
  etag: '7c26a0d7b72b6621bba1110da25d099e',
  url: 'http://res.cloudinary.com/hadar-staging/image/upload/v1482935950/smile.jpg',
  secure_url: 'https://res.cloudinary.com/hadar-staging/image/upload/v1482935950/smile.jpg',
  original_filename: 'smiling_man' }

Notification URL

You can tell Cloudinary to notify your application as soon as the upload completes by adding the notification_url parameter to the upload method and setting it to any valid HTTP or HTTPS URL. You also can set the notification_url parameter globally for all uploads on the Upload Settings page in the Cloudinary Management Console, instead of individually for each upload call.

Uploading New Files

Following a successful migration, you need to start uploading all new media assets to Cloudinary. There are several ways to do it: manually via the Cloudinary account console, calling the upload API, or using automatic upload. Another easy way to do this is using the ready-made upload widget.

Upload Widget

Cloudinary's upload widget includes a complete graphical interface. The widget supports a drag and drop functionality, interactive cropping, upload progress indication and thumbnail previews. The widget also monitors and handles uploading errors. The following code example shows how to open the widget:

<script src="//widget.cloudinary.com/global/all.js" type="text/javascript">  
</script>
<script>
  cloudinary.setCloudName('cld-name');
  cloudinary.openUploadWidget({upload_preset: 'my_preset'}, 
  function(error, result) {console.log(error, result)});
</script>

More information regarding the widget functionality is available here.

Summary

The steps detailed above are just the beginning of the journey of moving your media assets to the cloud. Once uploaded, Cloudinary supports a long list of image manipulation options, and the same goes for video. In addition, images can be optimized for faster delivery and support responsive design.

Now is the time to let go, send your media assets to the cloud and set them free.

Recent Blog Posts

Serverless Tutorial: File Storage with Webtask and Cloudinary

Media makes up the majority of today's website content. While it makes websites more interesting for visitors, media presents challenges because these images and videos are more difficult to store, deliver and manipulate on-the-fly to suit any given situation.

Read more

ImageCon17: Delivering Responsive Images

By Jason Grigsby

After five years many specifications, some inflamed Twitter battles and other conversations, responsive images have finally landed and there's a sound. Which is really exciting right? People have been climbing for this for quite some time and we've reached a point where they're available in modern browsers. So people were excited, they wanted to go use them it's something that designers and developers have had as a point of frustration for a long time.

Read more
Auto padding images with content-aware color padding

How you present the content of your website can be just as important as the content itself. The images you display need to conform to the graphic design of your site, and every image needs to fit within a predefined size. Although that may be simple enough to achieve when you are dealing with your own images, the task can be more challenging when displaying images uploaded by your users.

Read more

Bounce! Hacking Jazzfest with Social Videos

By Eric Normand
Bounce! Hacking Jazzfest with Social Videos

Last week, I was invited to an exclusive hackathon to build apps for musicians. The app team I was assigned to was tasked with building a video upload site for Bounce videos. Bounce is a style of music that originated in New Orleans. The app would be called BounceDotCom.com and there were plans to have Big Freedia, the Queen of Bounce, promote it. I knew the organizer could make things happen, so I jumped at the chance.

Read more
Getting a Better React-ion with Progressive Web Apps

This is part 2 of a 3 part series

React has become more popular, as well as more mature, over the last four years since its release by Facebook. It has become one of the go-to technologies for people looking to componentize the front-end of any web application. It also helps that an entire mobile stack is built around React in the form of ReactNative. The components are wonderful, however there can be a burdensome learning curve. But, in the end, there’s the payoff of highly reusable code and a better user experience.

Read more
Build an Image Library with React & Cloudinary

This article was originally posted on Scotch.io

React is a good tool when it comes to building flexible and reusable UI components. However, it's "one of those libraries" that cannot handle all the tasks involved in building a full fleshed UI project. Other supporting tools - such as a recently announced React SDK from Cloudinary - are available to provide solutions that the React core cannot.

Read more