Building Image Storyboard Apps on Android: A Tutorial

Globally, approximately two billion people now own smartphones, which also feature cameras capable of capturing photos and videos of a tonal richness and quality unimaginable even five years ago. Until recently, those cameras behaved mostly as optical sensors, catching light that determines the resulting image’s pixels. The next generation of cameras, however, can blend hardware and computer-vision algorithms that apply to an image’s semantic content, spawning creative mobile photo and video apps.

In early February, Google launched three mobile apps in their appsperiment scheme, which leverage such technologies as object recognition, person segmentation, image encoding and decoding, and garnering of user response. This tutorial focuses on one of those apps, Storyboard on Android, in which you transform videos into single-page comic layouts, all on devices. After you have selected a video and loaded it in Storyboard, the app picks interesting video frames, compresses JPEGs, and displays them on screen. You’ll learn how to replicate these features on Android with Cloudinary, a cloud-based, end-to-end solution for storing, optimizing, and delivering images and videos. Impressively, with Cloudinary, you can transform images and videos, that is, resize and add numerous effects to them.

So, to recap, , this tutorial walks you through the process of uploading a video to Cloudinary, in which you then generate, and deliver image thumbnails of any frame from the uploaded file, and render it on screen without having to compress the video. Furthermore, you will add effects to the generated image thumbnails and take them live through unique URLs.

As reference, see the Cloudinary post on working with videos on the computer or on mobile.

For details on image optimization and its background concepts, read this Cloudinary post.

This short clip shows you how Storyboard works. To run it on an Android device, download the app to your Google Drive.

While building the app, you might want to check out the source code.

As a prerequisite, set up a Cloudinary account.

Also,download Android Studio (or an IDE of your choice) and two third-party libraries:

• Android Studio
• Cloudinary Android SDK
• Picasso for asynchronous image loading

As a first step, open Android Studio and create an Android Studio project:

. Choose File > New > New Project.

In the **Create Android Project** dialog box, fill in the **Application name**, **Project location**, and **Package name** fields. Click **Next**.

![name the app and click next](https://cloudinary-res.cloudinary.com/image/upload/w_600,c_fill/dpr_auto/Named_App.jpg)

Under Select the form factors and minimum SDK in the Target Android Devices dialog box, select API 17: Android 4.2 (Jelly Bean). Click Next.Select target devices and API levels.

In the Add an Activity to Mobile dialog box, select Add No Activity. Click Next.

![select empty activity and click next](https://cloudinary-res.cloudinary.com/image/upload/w_600,c_fill/dpr_auto/Android_Activity.jpg)

In the Configure Activity dialog box, leave the default settings as is. Click Finish.

![leave the defaults and click finish](https://cloudinary-res.cloudinary.com/image/upload/w_600,c_fill/dpr_auto/Configure_Activity.jpg)

Next, add the code below to install Cloudinary and Picasso as a dependency in the build.gradle file of your app module:

    implementation group: 'com.cloudinary', name: 'cloudinary-android', version: '1.22.0'
    implementation 'com.squareup.picasso:picasso:2.5.2'
Code language: JavaScript (javascript)

Click Sync to install.

Afterwards, open the AndroidManifest.xml file and add Cloudinary configurations under the application tag:

    <manifest xmlns:android="http://schemas.android.com/apk/res/android"
       package="com.example.ekene.imageboardcloudinary">
    
        <uses-permission android:name="android.permission.INTERNET"/>
    
        <application
            ...
            >
            <meta-data
                android:name="CLOUDINARY_URL"
                android:value="cloudinary://@myCloudName"/>
        </application>
    </manifest>
Code language: HTML, XML (xml)

myCloudName is your Cloudinary name, which is on your console. Note that the Internet permissions are already in the manifest.

Now define the app layout with the following user-interface elements:

• A bar that shows the progress of the upload process
• A button that triggers the video upload
• Three ImageView objects that render the image thumbnails from Cloudinary

activity_main.xml View Activity main file

Next, initialize the view objects in the MainActivity.java file so you can refer to them later. Edit the file so it reads like this:

    package com.example.ekene.imageboardcloudinary;
    
    import ....
    
    public class MainActivity extends AppCompatActivity implements View.OnClickListener {
    
        private Button uploadBtn;
        private ProgressBar progressBar;
        private int SELECT_VIDEO = 2;
        private ImageView img1, img2, img3;
    
        @Override
        protected void onCreate(Bundle savedInstanceState) {
            super.onCreate(savedInstanceState);
            setContentView(R.layout.activity_main);
            img1 = findViewById(R.id.img1);
            img2 = findViewById(R.id.img2);
            img3 = findViewById(R.id.img3);
            progressBar = findViewById(R.id.progress_bar);
            uploadBtn = findViewById(R.id.uploadBtn);
        }
Code language: PHP (php)

Next, add the code below to set up the Upload button, which, on a click, launches the gallery in which you can select a video for upload

    uploadBtn.setOnClickListener(new View.OnClickListener() {
        @Override
        public void onClick(View view) {
            pickVideoFromGallery();
        }
        private void pickVideoFromGallery() {
            Intent GalleryIntent = new Intent();
            GalleryIntent.setType("video/*");
            GalleryIntent.setAction(Intent.ACTION_GET_CONTENT);
            startActivityForResult(Intent.createChooser(GlleryIntent, 
            "select video"), SELECT_VIDEO);
        }
    });
Code language: JavaScript (javascript)

Subsequently, as soon as you select a video, Cloudinary calls the onActivityResult method, which in turn triggers an upload to Cloudinary. To facilitate uploads, create an UploadRequest method and dispatch it within the onActivityResult method, as described in the next section..

Cloudinary offers two types of uploads:

• Signed uploads, which require an authentication signature from a backend. For these uploads, your images and videos are signed with the API and secret key in the console. Because those signatures are risky on the client side, which can be easily decompiled, a back-end is mandatory for security.

• Unsigned uploads, which do not require signatures and are thus less secure than signed ones. These uploads are controlled by an upload preset, which defines the options that apply to the images that are uploaded with the preset.

Here, you set up unsigned uploads by enabling them on the console: Select Settings on your dashboard, select the Upload tab, scroll down to Upload presets, and enable Unsigned. Cloudinary then generates a preset with a random string as its name. Copy the name and set it aside for use later.

Next, enable the upload of a selected video to Cloudinary by first initializing the Cloudinary MediaManager class onCreate method in the MainActivity class, as shown here:

    package com.example.ekene.imageboardcloudinary;
    import ....
    
    public class MainActivity extends AppCompatActivity implements View.OnClickListener {
        @Override
        protected void onCreate(Bundle savedInstanceState) {
            super.onCreate(savedInstanceState);
            setContentView(R.layout.activity_main);
            //initialize MediaManager
            MediaManager.init(this);
        }
Code language: JavaScript (javascript)

Subsequently, call the onActivityResult() method in MainActivity and set it up.

As background, here are the methods in MainActivity this app has adopted so far and their relationships to onActivityResult:

• onCreate — Becomes active when the app starts. The XML file in use earlier is the default layout of the activity. You also added a listener to the button, which, on a click, causes the app to call the second method, pickVideoFromGallery (see below).

• pickVideoFromGallery — Launches the user’s gallery for video selection. This process generates a unique request code in the SELECT_VIDEO variable. Because a response follows a video section, the startActivityForResult() method renders the response on the onActivityForResult() method. If the selection succeeds, so does the response, and the selectedVideo variable in turn holds URI of the selected video for upload. Finally, Culinary calls the onActivityForResult() method (see below). If the selection fails, the process ends.

• onActivityForResult() — Checks if the response from startActivityForResult() succeeded. On a video selection, resultCode is equal to Activity.RESULT_OK; otherwise, to Activity.RESULT_CANCELLED. A success results in an upload request to Cloudinary with MediaManager, like this:

    @Override
    protected void onActivityResult(int requestCode, int resultCode, final Intent data) {
    
        if (requestCode == SELECT_VIDEO && resultCode == RESULT_OK) {
            Uri selectedVideo = data.getData();
            //...
            }
        }
Code language: PHP (php)

Now create an upload request by adding MediaManager to onActivityResult and build UploadRequest in Cloudinary with the following five methods:

• upload() — Takes in uri of the selected video for uploads.
• unsigned() – Takes in preset name from your console.
• option() – Takes in resource_type of the upload.

    MediaManager.get()
                    .upload(selectedVideo)
                    .unsigned("preset_name")
                    .option("resource_type", "video")
                    .callback(...)
                   //...
                }
Code language: JavaScript (javascript)

• callback() — Takes in a new UploadCallback method, which implements several other callback methods that track the progress of the upload.
• onStart() — Defines what happens when upload starts:

    .callback(new UploadCallback() {
                        @Override
                        public void onStart(String requestId) {
                            progressBar.setVisibility(View.VISIBLE);
                            Toast.makeText(MainActivity.this, 
                            "Upload has started...", Toast.LENGTH_SHORT).show();
                        }
                        //...
Code language: JavaScript (javascript)

The above code makes progressBar visible and sets Toast, which displays the message “Upload has started.”

• onProgress() — Defines what happens as the upload progresses, that is, has started but not yet completed. Leave it blank in this app::

    .callback(new UploadCallback() {
                       @Override
                        public void onProgress(...) {
                        }
                        //...
Code language: JavaScript (javascript)

• onSuccess() — Defines what happens if the video upload succeeds. In this app, you generate the required images from the uploaded video and pass them to the ImageView object with Piccasso:

    .callback(new UploadCallback() {
                        @Override
                        public void onSuccess(String requestId, Map resultData) {
                            Toast.makeText(MainActivity.this, 
                            "Uploaded Succesfully", Toast.LENGTH_SHORT).show();
                            progressBar.setVisibility(View.GONE);
                            uploadBtn.setVisibility(View.INVISIBLE);
                            // on successful upload, we start generating the images
                            // first get the video unique id
                            String publicId = resultData.get("public_id").toString();
                            //generate the first image from the id 
                            String firstImgUrl = MediaManager.get().url().transformation(                        new Transformation().startOffset("12").border(
                            "5px_solid_black").border("5px_solid_black")).resourceType(
                            "video").generate(publicId+".jpg");                        
                            // load the first image into the image view
                            Picasso.with(getApplicationContext()).load(
                            firstImgUrl).into(img1);
                        }
                        //...
Code language: JavaScript (javascript)

The above onSuccess() method outputs requestId and details from the upload. You then access the uploaded video’s URL by calling resultData.get("url") and the video’s ID by calling resultData.get("public_id"). Subsequently, generate the images from the video ID and store their URLs in a string variable.

Afterwards, load the images into their image view objects with Picasso.

• onError() – Defines what happens if an error occurs during the upload process. For this app, create a Toast method for the error message, like this:

    .callback(new UploadCallback() {
                        @Override
                        public void onError(String requestId, ErrorInfo error) {
                            Toast.makeText(MainActivity.this, 
                            "Upload Error", Toast.LENGTH_SHORT).show();
                            Log.v("ERROR!!", error.getDescription());
                        }
                        //...
Code language: JavaScript (javascript)

• dispatch() — Defines when and how each upload request runs.

You have now set up the onActivityForResult() method. When it receives a response from pickVideoFromGallery(), it creates a request to upload to Cloudinary with the URL of the selected video. Once the upload succeeds, onActivityForResult() generates image thumbnails and renders them on the ImageView objects, which you defined earlier.

In case of errors, have a look at the source code. Long code lines can be overwhelming sometimes.

Once you have generated the image thumbnails, you can further manipulate them as any other images. For example, you generated a cropped thumbnail from the third image with a width of 200 pixels and a height of 150 pixels on the frame at 20 seconds. Also, the image is rendered in grayscale with rounded corners and a five-pixel-wide black border. Here is the code in question:

    String thirdImgUrl = MediaManager.get().url().transformation
                            (new Transformation().startOffset("20").width(200).height(150)
                            .radius(20).effect("grayscale")
                            .border("5px_solid_black").crop("crop")).resourceType("video")
                            .format("jpg")
    .generate(publicId);
Code language: JavaScript (javascript)

For diversity, different manipulations apply to each of the three images. That’s an excellent approach to adopt so as to institute responsiveness in images.

This tutorial highlights only a fraction of Cloudinary’s capabilities for transforming images. To learn more, see the related section in the Cloudinary documentation.

You are now up to speed with Cloudinary’s outstanding capabilities for transforming images and videos. For details on Cloudinary and its vast array of useful services, check out the documentation.