Cloudinary Blog

MythBusters, Part 2: Automated UI Tests Are Slow

Alexey Kagansky
By Alexey Kagansky
How to Speed Up Automated UI Tests

Part 1 of MythBusters busted the myth of unstable tests. Part 2 here smashes another myth: “UI tests are slow.” Far be it from me to dispute that UI tests are slower than unit or integration tests, but let me show you the best practices that speed up UI tests.

Let's first consider the steps you’d take for a typical, simple test:

  1. Log in to the system.
  2. Prepare the data for the test.
  3. Validate that the used data appears as required.

To execute a slow test with Selenium, you’d take these three steps:

  1. To log in, you must navigate to the home page, click the login icon, and type in your credentials.
  2. To prepare the data for the test, you must create the data in the UI.
  3. To validate the data, you must take a few steps.

Depending on the speed of the network and database usage, such a typical test at Cloudinary takes 20-30 seconds.

SETUP TEST DATA VIA API AND NOT UI

However, that same test in our test suite takes only 10 seconds, a 50-percent reduction in time spent. That might seem insignificant, but if there are thousands of tests, those seconds add up, resulting in many expensive minutes. Here are the steps we take to accelerate the process:

  1. For logins, we use an authentication API so that we are automatically logged in to the system after navigating to the home page.
  2. For data preparation, we also use an API. For example, in our automation framework, we upload media assets with SDK calls.
  3. For validation, we follow the same steps as those in the “long” test.

As to the parallelism of test execution, many frameworks offer parallel execution out of the box. To prepare for that, we do the following:

  1. Write independent tests. Our tests must be independent. Dependent tests might run in parallel, but they would fail due to the dependencies. For example, in case of parallel execution, Test 2 might run before Test 1, but if Test 1 depends on Test 2, Test 1 will fail.

  2. Use parallel test runners. We must prepare solutions for continuous integration (CI) by leveraging free tools like Selenium Grid and Selenoid, or paid solutions. We can also run tests in parallel with the framework without any third-party software.

  3. Aim for cost-effectiveness. Because parallel execution requires more CPU and memory, we must boost the performance of the executing machines, i.e., find the sweet spot between lab size and test execution time.

    At Cloudinary, we use five instances of AWS and execute 10 parallel test runners.

  4. Optimize the test size. Another important factor that affects time performance is the test size. Tests should be short and should check only what is required. In manual testing, we can check many details along the way, e.g., the login process, links, colors, fonts, etc. However, in an automated test, we need to check only one thing.

    Also, remember that not every manual test needs to be automated. In case of bugs, if your automation framework is built correctly, you will see only the failures that relate to those bugs, which is a big help for debugging.

automated tests
At Cloudinary, we follow all the rules described above for over 1,000 automated tests with a total execution time of about 40 minutes. We don't stop there, however, and will continue to look for ways to save even more time in automated tests.

Recent Blog Posts

Create Lightweight Sites With Low-Code and No-Code Technology

Consumers expect modern websites to be mainly visual. But, the more compelling and complex the related media is, the more data is involved, compounding the site’s weight. In today’s content-craving world, delivering unoptimized media can cost you because it leads to sluggish page loads, resulting in visitors abandoning your site in search of a faster alternative. In fact, a page load that takes more than three seconds can cause as many as 40% of your visitors to bounce. Given this competitive, digital-first environment, you can’t afford to lose page views, for time is of the essence.

Read more
A Blueprint for AWS-Secured Webhook Listeners for Cloudinary

tl;dr: An AWS-secured and optimized Cloudinary webhook listener for extending the Cloudinary service

Code: Github

A webhook is a communication medium for sending notifications from one platform to another about events that occurred. In place are user-defined HTTP callbacks that are triggered by specific events. When a triggered event takes place on the source site, the webhook listens to the event, collects the data, and sends it to the URL you specified in the form of an HTTP request.

Read more
New Accessibility Features for Cloudinary’s Product Gallery Widget

Cloudinary’s Product Gallery widget, which launched in 2019, has enabled many brands to effectively and efficiently showcase their products in a sleek and captivating manner, saving countless hours of development time and accelerating release cycles. By adding Cloudinary’s Product Gallery widget with its customizable UI to their product page, retailers reap numerous benefits, often turning visitors into customers in short order.

Read more
Why Successful Businesses Engage With and Convert Audiences With Visual Media

Most business buyers prefer to research purchase options online, as do many shoppers. No wonder online retail sales in the U.S. rose by 32.4% in 2020—an impressive gain of $105 billion.

For B2B and B2C businesses, text-heavy websites are no longer adequate in attracting shoppers. Instead, engaging visual media—spin images, videos, 3D models, augmented reality—are becoming a must for conveying eye-catching details and differentiators about products or services.

Read more
Making User-Generated Content (UGC) Shoppable With Cloudinary

User-generated content (UGC) is a powerful marketing tool. Not only does video complement marketing efforts for e-commerce by enabling customers to explore products in greater detail, but UGC also adds an element of trust. As a bonus, user-generated video is an exceptional opportunity for e-businesses to attract website traffic without their marketing team having to create promotional videos from scratch. User-generated content drives conversions and brand loyalty as a direct result of authentic interaction.

Read more