Cloudinary Blog

Testing Functional UI the Cloudinary Way

Automated UI Testing with End-to-End JavaScript Tests

You may have heard that we recently launched our Digital Asset Management (DAM) system. On top of other challenges, that was our first launch targeted at the nondeveloper audience, to whom the UI is a central element of a product’s capabilities.

Releasing robust products on time require comprehensive automated testing at all levels: Unit , API, and UI testing. This post describes our solution for end-to-end UI automation testing, which serves both QA engineers and front-end developers in minimizing bugs in releases and in speeding up testing for faster development and shorter release cycles.

Objectives

Right off the bat, we had these objectives in mind:

  • Create a JavaScript-based test automation framework with Node.js so that developers can stay in their “comfort zone.”
  • Facilitate root-cause error analysis through informative reports for a thorough investigation of failed tests. The reports must be human friendly, containing as many relevant details as possible, such as logs, screenshots, and page sources.
  • Execute in parallel for fast, continuous integration feedback.
  • Build configurable test suites for various platforms and setups (deployment, pull requests, on demand, and such).
  • Enhance browser coverage by running tests on various browsers with local and remote setups.
  • Develop reusable components with a functional interface that mimics user actions and that supports test maintenance and readability.

Implementation

Let’s now introduce our new test automation framework wdio-allure-ts, which promises to help you quickly start authoring end-to-end JavaScript ui tests with a useful, informative report.

First, some background: We considered and tested many other leading testing solutions, such as Allure Reporter, which offers a pleasing and functional design. You can also easily integrate it with Node.js and Jenkins. However, Allure Reporter falls short in several ways:

  • The Reporter logs WebDriver commands (GET and POST) that give no clues on failures.
  • Page-source and browser-console logs of failed tests are absent in the reports.
  • Most of the errors reflect timeouts only. No clear failure reasons. WebdriverIO, a Node.js testing utility, meets our needs. With a sizable community coupled with simple setups for configuration and customization, WebdriverIO supports third-party integrations (automation testing reports, test runners), Page Object Model, and synchronous execution. Also, we chose TypeScript instead of plain JavaScript for IntelliSense support, which spells fast and seamless development with typed variables.. Subsequently, we blended those tools into our new, open-source solution for end-to-end functional testing: wdio-allure-ts.That solution wraps the most common WebdriverIO actions, generating intuitive error messages in case of failure, custom logs for the Allure Reporter, more validations for enhanced stability, and last, but not least, IntelliSense.

Example With Pure WebdriverIO

Now take a look at an example of an action that, after validating that a particular element is visible, clicks it, logs every step to the Reporter, and throws meaningful errors for failures, if any.

   const selector: string = "someSelector";
   logger(`Click an element with selector: ${selector}`);
   try {
     logger(`Validate element with selector ${selector} is visible`);
     browser.isVisible(selector);
   } catch (error) {
     throw new Error(`Tried to click not visible element, ${error}`);
   }

   try {
     logger("Perform click action");
     browser.click(selector);
   } catch (error) {
     throw new Error(
       `Failed to click an element by given selector ${selector}. ${error}`
     );
   }

Example with wdio-allure-ts:

const selector: string = "someSelector";
BrowserUtils.click(selector);

You can see that our new framework offers the same capabilities with much cleaner code. Because the framework automatically handles logging and error reporting, automation developers can focus on testing the business logic. You can add more report logs with a simple Reporter API for log levels: step, debug, error, info, and warning. The logs are displayed on the terminal and reflected in the report. Example:

import { Reporter } from 'wdio-allure-ts';
Reporter.step('Step log entry');
Reporter.error('Error log entry');

Terminal Output

Terminal Output
Note the difference in the highlighted areas between the original report and the one created with wdio-allure-ts. Original Report

Original Report

wdio-allure-ts Report(live report example):

`wdio-allure-ts` Report

That's It. Ready to Try?

For you to take a crack at it, we've created a sample project with a quick introduction to our framework and its usage on a real-world application The project contains examples for the following:

Just clone the repo, read the README, and create new tests according to the examples. We'd love to hear your thoughts and ideas for integrating wdio-allure-ts into your testing workflow. Send them to us in the Comments section below please.

Recent Blog Posts

 New Image File Format: FUIF:Lossy, Lossless, and Free

I've been working to create a new image format, which I'm calling FUIF, or Free Universal Image Format. That’s a rather pretentious name, I know. But I couldn’t call it the Free Lossy Image Format (FLIF) because that acronym is not available any more (see below) and FUIF can do lossless, too, so it wouldn’t be accurate either.

Read more
Optimizing Video Streaming and Delivery: Q&A with Doug Sillars

Doug Sillars, a digital nomad and a freelance mobile-performance expert, answers questions about video streaming and delivery, website optimization, and more.

Doug Sillars, a freelance mobile-performance expert and developer advocate, is a Google Developer Expert and the author of O’Reilly’s High Performance Android Apps. Given his extensive travels across the globe—from the UK to Siberia—with his wife, kids, and 11-year-old dog, Max, he has been referred to as a “digital nomad.” So far in 2018, Doug has spoken at more than 75 meetups and conferences!

Read more
Building a Music Discovery Service

In May 2018, Cloudinary sponsored Capitol Music Group’s first hackathon held by its new Capitol360 Innovation Center, which aims at connecting musicians and software technologists to facilitate and stimulate the creation of music. See this interview for details. As a starter project for the hackathon, we built a sample app called Music Discovery Service.

Read more
Once a Hackathon Participant, Now a Judge

Over the past several years, I've had a chance to participate in hackathons, as part of teams that developed a social payment app and helped users decide meals to cook. But it wasn't until last month that I got to experience a hackathon from the other side, as a judge.

Read more
10 Website Videos Mistakes and How to Solve Them

It should come as no surprise that video use on the internet is exploding. You can see the dramatic growth of video on the average site in this SpeedCurve blog post.

With the growth in video comes greater bandwidth use, which is not only costly for your IT budget, but for your visitors as well. Beyond the expense, there is the user experience to consider. The heavier the page, the longer it will take to load, and the greater likelihood visitors will abandon your site. Page load speed is also an important factor in SEO ranking, so clearly video is something we need to take seriously and get right. Video is challenging, presenting terms still unfamiliar to developers - like codecs, bitrate and adaptive bitrate streaming. As a result, mistakes are being made in video implementation.

Read more
Android Data Saver: Optimizing Mobile Data Usage with Cloudinary

Over the life of a mobile device, the cost of a cellular data plan often exceeds that of the device itself. To optimize data usage and purge useless data on their mobile devices, users can enable Data Saver from Android 7.0 (API level 24). To do so, users toggle Data Saver in quick settings under the Notification shade or under Settings > Data usage. With Data Saver enabled, apps that aren't whitelisted cannot use cellular data in the background. They are also directed to consume less data while active.

Read more