Cloudinary Blog

Building Image Storyboard Apps on Android: A Tutorial

Build Your Own Image Storyboard Android App

Globally, approximately two billion people now own smartphones, which also feature cameras capable of capturing photos and videos of a tonal richness and quality unimaginable even five years ago. Until recently, those cameras behaved mostly as optical sensors, catching light that determines the resulting image's pixels. The next generation of cameras, however, can blend hardware and computer-vision algorithms that apply to an image's semantic content, spawning creative mobile photo and video apps.

In early February, Google launched three mobile apps in their appsperiment scheme, which leverage such technologies as object recognition, person segmentation, image encoding and decoding, and garnering of user response. This tutorial focuses on one of those apps, Storyboard on Android, in which you transform videos into single-page comic layouts, all on devices. After you have selected a video and loaded it in Storyboard, the app picks interesting video frames, compresses JPEGs, and displays them on screen. You’ll learn how to replicate these features on Android with Cloudinary, a cloud-based, end-to-end solution for storing, optimizing, and delivering images and videos. Impressively, with Cloudinary, you can transform images and videos, that is, resize and add numerous effects to them.

So, to recap, , this tutorial walks you through the process of uploading a video to Cloudinary, in which you then generate, and deliver image thumbnails of any frame from the uploaded file, and render it on screen without having to compress the video. Furthermore, you will add effects to the generated image thumbnails and take them live through unique URLs.

As reference, see the Cloudinary post on working with videos on the computer or on mobile.

For details on image optimization and its background concepts, read this Cloudinary post.

Demo and Source Code

This short clip shows you how Storyboard works. To run it on an Android device, download the app to your Google Drive.

While building the app, you might want to check out the source code.

Requirements

As a prerequisite, set up a Cloudinary account.

Also,download Android Studio (or an IDE of your choice) and two third-party libraries:

Setup of Android

As a first step, open Android Studio and create an Android Studio project:

. Choose File > New > New Project.

New Project

In the **Create Android Project** dialog box, fill in the **Application name**, **Project location**, and **Package name** fields. Click **Next**.

![name the app and click next](https://cloudinary-res.cloudinary.com/image/upload/w_600,c_fill/dpr_auto/Named_App.jpg)

Under Select the form factors and minimum SDK in the Target Android Devices dialog box, select API 17: Android 4.2 (Jelly Bean). Click Next.Select target devices and API levels.

set minimum sdk to API 17 and click next

In the Add an Activity to Mobile dialog box, select Add No Activity. Click Next.

![select empty activity and click next](https://cloudinary-res.cloudinary.com/image/upload/w_600,c_fill/dpr_auto/Android_Activity.jpg)

In the Configure Activity dialog box, leave the default settings as is. Click Finish.

![leave the defaults and click finish](https://cloudinary-res.cloudinary.com/image/upload/w_600,c_fill/dpr_auto/Configure_Activity.jpg)

Setup of Dependencies

Next, add the code below to install Cloudinary and Picasso as a dependency in the build.gradle file of your app module:

    implementation group: 'com.cloudinary', name: 'cloudinary-android', version: '1.22.0'
    implementation 'com.squareup.picasso:picasso:2.5.2'

Click Sync to install.

Afterwards, open the AndroidManifest.xml file and add Cloudinary configurations under the application tag:

    <manifest xmlns:android="http://schemas.android.com/apk/res/android"
       package="com.example.ekene.imageboardcloudinary">

        <uses-permission android:name="android.permission.INTERNET"/>

        <application
            ...
            >
            <meta-data
                android:name="CLOUDINARY_URL"
                android:value="cloudinary://@myCloudName"/>
        </application>
    </manifest>

myCloudName is your Cloudinary name, which is on your console. Note that the Internet permissions are already in the manifest.

App Layout

Now define the app layout with the following user-interface elements:

  • A bar that shows the progress of the upload process
  • A button that triggers the video upload
  • Three ImageView objects that render the image thumbnails from Cloudinary

activity_main.xml View Activity main file

Next, initialize the view objects in the MainActivity.java file so you can refer to them later. Edit the file so it reads like this:

    package com.example.ekene.imageboardcloudinary;

    import ....

    public class MainActivity extends AppCompatActivity implements View.OnClickListener {

        private Button uploadBtn;
        private ProgressBar progressBar;
        private int SELECT_VIDEO = 2;
        private ImageView img1, img2, img3;

        @Override
        protected void onCreate(Bundle savedInstanceState) {
            super.onCreate(savedInstanceState);
            setContentView(R.layout.activity_main);
            img1 = findViewById(R.id.img1);
            img2 = findViewById(R.id.img2);
            img3 = findViewById(R.id.img3);
            progressBar = findViewById(R.id.progress_bar);
            uploadBtn = findViewById(R.id.uploadBtn);
        }

Next, add the code below to set up the Upload button, which, on a click, launches the gallery in which you can select a video for upload

    uploadBtn.setOnClickListener(new View.OnClickListener() {
        @Override
        public void onClick(View view) {
            pickVideoFromGallery();
        }
        private void pickVideoFromGallery() {
            Intent GalleryIntent = new Intent();
            GalleryIntent.setType("video/*");
            GalleryIntent.setAction(Intent.ACTION_GET_CONTENT);
            startActivityForResult(Intent.createChooser(GlleryIntent, 
            "select video"), SELECT_VIDEO);
        }
    });

Subsequently, as soon as you select a video, Cloudinary calls the onActivityResult method, which in turn triggers an upload to Cloudinary. To facilitate uploads, create an UploadRequest method and dispatch it within the onActivityResult method, as described in the next section..

Cloudinary Uploads

Cloudinary offers two types of uploads:

  • Signed uploads, which require an authentication signature from a backend. For these uploads, your images and videos are signed with the API and secret key in the console. Because those signatures are risky on the client side, which can be easily decompiled, a back-end is mandatory for security.

  • Unsigned uploads, which do not require signatures and are thus less secure than signed ones. These uploads are controlled by an upload preset, which defines the options that apply to the images that are uploaded with the preset.

Here, you set up unsigned uploads by enabling them on the console: Select Settings on your dashboard, select the Upload tab, scroll down to Upload presets, and enable Unsigned. Cloudinary then generates a preset with a random string as its name. Copy the name and set it aside for use later.

unsigned

Next, enable the upload of a selected video to Cloudinary by first initializing the Cloudinary MediaManager class onCreate method in the MainActivity class, as shown here:

    package com.example.ekene.imageboardcloudinary;
    import ....

    public class MainActivity extends AppCompatActivity implements View.OnClickListener {
        @Override
        protected void onCreate(Bundle savedInstanceState) {
            super.onCreate(savedInstanceState);
            setContentView(R.layout.activity_main);
            //initialize MediaManager
            MediaManager.init(this);
        }

Subsequently, call the onActivityResult() method in MainActivity and set it up.

As background, here are the methods in MainActivity this app has adopted so far and their relationships to onActivityResult:

  • onCreate — Becomes active when the app starts. The XML file in use earlier is the default layout of the activity. You also added a listener to the button, which, on a click, causes the app to call the second method, pickVideoFromGallery (see below).

  • pickVideoFromGallery — Launches the user’s gallery for video selection. This process generates a unique request code in the SELECT_VIDEO variable. Because a response follows a video section, the startActivityForResult() method renders the response on the onActivityForResult() method. If the selection succeeds, so does the response, and the selectedVideo variable in turn holds URI of the selected video for upload. Finally, Culinary calls the onActivityForResult() method (see below). If the selection fails, the process ends.

  • onActivityForResult() — Checks if the response from startActivityForResult() succeeded. On a video selection, resultCode is equal to Activity.RESULT_OK; otherwise, to Activity.RESULT_CANCELLED. A success results in an upload request to Cloudinary with MediaManager, like this:

    @Override
    protected void onActivityResult(int requestCode, int resultCode, final Intent data) {

        if (requestCode == SELECT_VIDEO && resultCode == RESULT_OK) {
            Uri selectedVideo = data.getData();
            //...
            }
        }

Now create an upload request by adding MediaManager to onActivityResult and build UploadRequest in Cloudinary with the following five methods:

  • upload() — Takes in uri of the selected video for uploads.
  • unsigned() – Takes in preset name from your console.
  • option() – Takes in resource_type of the upload.
    MediaManager.get()
                    .upload(selectedVideo)
                    .unsigned("preset_name")
                    .option("resource_type", "video")
                    .callback(...)
                   //...
                }
  • callback() — Takes in a new UploadCallback method, which implements several other callback methods that track the progress of the upload.
  • onStart() — Defines what happens when upload starts:
    .callback(new UploadCallback() {
                        @Override
                        public void onStart(String requestId) {
                            progressBar.setVisibility(View.VISIBLE);
                            Toast.makeText(MainActivity.this, 
                            "Upload has started...", Toast.LENGTH_SHORT).show();
                        }
                        //...

The above code makes progressBar visible and sets Toast, which displays the message “Upload has started.”

  • onProgress() — Defines what happens as the upload progresses, that is, has started but not yet completed. Leave it blank in this app::
    .callback(new UploadCallback() {
                       @Override
                        public void onProgress(...) {
                        }
                        //...
  • onSuccess() — Defines what happens if the video upload succeeds. In this app, you generate the required images from the uploaded video and pass them to the ImageView object with Piccasso:
    .callback(new UploadCallback() {
                        @Override
                        public void onSuccess(String requestId, Map resultData) {
                            Toast.makeText(MainActivity.this, 
                            "Uploaded Succesfully", Toast.LENGTH_SHORT).show();
                            progressBar.setVisibility(View.GONE);
                            uploadBtn.setVisibility(View.INVISIBLE);
                            // on successful upload, we start generating the images
                            // first get the video unique id
                            String publicId = resultData.get("public_id").toString();
                            //generate the first image from the id 
                            String firstImgUrl = MediaManager.get().url().transformation(                        new Transformation().startOffset("12").border(
                            "5px_solid_black").border("5px_solid_black")).resourceType(
                            "video").generate(publicId+".jpg");                        
                            // load the first image into the image view
                            Picasso.with(getApplicationContext()).load(
                            firstImgUrl).into(img1);
                        }
                        //...

The above onSuccess() method outputs requestId and details from the upload. You then access the uploaded video’s URL by calling resultData.get("url") and the video’s ID by calling resultData.get("public_id"). Subsequently, generate the images from the video ID and store their URLs in a string variable.

Afterwards, load the images into their image view objects with Picasso.

  • onError() - Defines what happens if an error occurs during the upload process. For this app, create a Toast method for the error message, like this:
    .callback(new UploadCallback() {
                        @Override
                        public void onError(String requestId, ErrorInfo error) {
                            Toast.makeText(MainActivity.this, 
                            "Upload Error", Toast.LENGTH_SHORT).show();
                            Log.v("ERROR!!", error.getDescription());
                        }
                        //...
  • dispatch() — Defines when and how each upload request runs.

You have now set up the onActivityForResult() method. When it receives a response from pickVideoFromGallery(), it creates a request to upload to Cloudinary with the URL of the selected video. Once the upload succeeds, onActivityForResult() generates image thumbnails and renders them on the ImageView objects, which you defined earlier.

In case of errors, have a look at the source code. Long code lines can be overwhelming sometimes.

Transformation

Once you have generated the image thumbnails, you can further manipulate them as any other images. For example, you generated a cropped thumbnail from the third image with a width of 200 pixels and a height of 150 pixels on the frame at 20 seconds. Also, the image is rendered in grayscale with rounded corners and a five-pixel-wide black border. Here is the code in question:

    String thirdImgUrl = MediaManager.get().url().transformation
                            (new Transformation().startOffset("20").width(200).height(150)
                            .radius(20).effect("grayscale")
                            .border("5px_solid_black").crop("crop")).resourceType("video")
                            .format("jpg")
    .generate(publicId);

For diversity, different manipulations apply to each of the three images. That’s an excellent approach to adopt so as to institute responsiveness in images.

This tutorial highlights only a fraction of Cloudinary’s capabilities for transforming images. To learn more, see the related section in the Cloudinary documentation.

Conclusion

You are now up to speed with Cloudinary’s outstanding capabilities for transforming images and videos. For details on Cloudinary and its vast array of useful services, check out the documentation.

Recent Blog Posts

Integrating Cloudinary with Forestry’s Media Library

At Forestry, we believe that there is a bright future for static HTML sites built with tools like Jekyll and Hugo. These tools can create sites that run well, and are easy to host and maintain, because they don’t require any server-side code.

Read more
Video Optimization With the HTML5 <video> Player

Lack of experience and compression knowhow can cause significant user-experience problems. For instance, on a major retail site, I recently ran into a 48 MB video-hero banner. Pulling out the video and encoding it as an H.264 MP4 reduces the size to 1.9 MB. So, despite the desire for more video content, developers have not yet caught up to best practices. How do we get the best of both worlds without creating a disaster like the one above?

Read more
Build a Facial Emotion Recognition Based Video Suggestion App

Developers are always looking for new and creative ways to deliver content that resonates with the way users feel. Often using the latest technical innovations the market has to offer such as Artificial Intelligence (AI) and Machine Learning (ML). What better way to demonstrate innovative uses of these technology in a consumer market than capturing expressions from your users and then serving content based on that expression!

Read more
Improve Customer Data Protection with GDPR Implementation

TL;DR

Yay! We've done it! Gold-Star for us! We've talked with all the people, made all the changes, paid all the lawyers and checked all the boxes. GDPR? ✅Done!

Not so fast. Of course, conforming to the GDPR regulations introduced in Europe is just the beginning. This is a process and a state of mind that must become part of our long-term cultural ethos.

Read more
Magento Image and Video Optimization

As the number of channels and devices continues to grow, it is becoming much more challenging to deliver an optimal visual experience. On an eCommerce site, engagement and conversions are critical. Factors such as page load time strongly influence search engine rankings, shopping experiences, conversion rates and, ultimately, your revenue. An optimized visual experience positively affects conversion rates. But it also can introduce resource bottlenecks, as every image and video needs to be delivered in the most efficient format, quality and resolution, based on the viewing device.

Read more