Cloudinary Blog

MythBusters, Part 2: Automated UI Tests Are Slow

Alexey Kagansky
By Alexey Kagansky
How to Speed Up Automated UI Tests

Part 1 of MythBusters busted the myth of unstable tests. Part 2 here smashes another myth: “UI tests are slow.” Far be it from me to dispute that UI tests are slower than unit or integration tests, but let me show you the best practices that speed up UI tests.

Let's first consider the steps you’d take for a typical, simple test:

  1. Log in to the system.
  2. Prepare the data for the test.
  3. Validate that the used data appears as required.

To execute a slow test with Selenium, you’d take these three steps:

  1. To log in, you must navigate to the home page, click the login icon, and type in your credentials.
  2. To prepare the data for the test, you must create the data in the UI.
  3. To validate the data, you must take a few steps.

Depending on the speed of the network and database usage, such a typical test at Cloudinary takes 20-30 seconds.

SETUP TEST DATA VIA API AND NOT UI

However, that same test in our test suite takes only 10 seconds, a 50-percent reduction in time spent. That might seem insignificant, but if there are thousands of tests, those seconds add up, resulting in many expensive minutes. Here are the steps we take to accelerate the process:

  1. For logins, we use an authentication API so that we are automatically logged in to the system after navigating to the home page.
  2. For data preparation, we also use an API. For example, in our automation framework, we upload media assets with SDK calls.
  3. For validation, we follow the same steps as those in the “long” test.

As to the parallelism of test execution, many frameworks offer parallel execution out of the box. To prepare for that, we do the following:

  1. Write independent tests. Our tests must be independent. Dependent tests might run in parallel, but they would fail due to the dependencies. For example, in case of parallel execution, Test 2 might run before Test 1, but if Test 1 depends on Test 2, Test 1 will fail.

  2. Use parallel test runners. We must prepare solutions for continuous integration (CI) by leveraging free tools like Selenium Grid and Selenoid, or paid solutions. We can also run tests in parallel with the framework without any third-party software.

  3. Aim for cost-effectiveness. Because parallel execution requires more CPU and memory, we must boost the performance of the executing machines, i.e., find the sweet spot between lab size and test execution time.

    At Cloudinary, we use five instances of AWS and execute 10 parallel test runners.

  4. Optimize the test size. Another important factor that affects time performance is the test size. Tests should be short and should check only what is required. In manual testing, we can check many details along the way, e.g., the login process, links, colors, fonts, etc. However, in an automated test, we need to check only one thing.

    Also, remember that not every manual test needs to be automated. In case of bugs, if your automation framework is built correctly, you will see only the failures that relate to those bugs, which is a big help for debugging.

automated tests
At Cloudinary, we follow all the rules described above for over 1,000 automated tests with a total execution time of about 40 minutes. We don't stop there, however, and will continue to look for ways to save even more time in automated tests.

Recent Blog Posts

Automatically Translating Videos for an International Audience

No matter your business focus—public service, B2B integration, recruitment—multimedia, in particular video, is remarkably effective in communicating with the audience. Before, making video accessible to diverse viewers involved tasks galore, such as eliciting the service of production studios to manually dub, transcribe, and add subtitles. Those operations were costly and slow, especially for globally destined content.

Read more
Cloudinary Helps Minted Manage Its Image-Generation Pipeline at Scale

David first shared his thoughts at our ImageCon coverence last October and this case study is an abbreviated version of Minted’s success using Cloudinary.

Over time, Faithful renderings of the creations of the illustrators, textile designers, painters, packaging designers, marketers, and stay-at-home moms, all of whom are core contributors of the Minted world, was getting harder and harder. Legacy technology wasn’t cutting it any more—and it was time for Cloudinary to step in.

Read more
Highlights on ImageCon 2021 and a Preview of ImageCon 2022

New year, same trend! Visual media will continue to play a monumental role in driving online conversions. To keep up with visual-experience trends and best practices, Cloudinary holds an annual conference called ImageCon, a one-of-a-kind event that helps attendees create the most engaging visual experiences possible.

Read more

New for DAM: Media Library Extension for Chrome

By Sharon Yelenik
A New Media Library Chrome Extension for Cloudinary DAM

With the introduction of the Media Library Extension, a Chrome-browser add-on that streamlines the access to, search for, and management of images and videos, Cloudinary offers yet another effective tool for its Digital Asset Management (DAM) solution. Let’s have a look at how most teams are currently working with media assets and how the new add-on not only boosts efficiency, but also renders the process a pleasure to work with.

Read more
New Features Supercharge Cloudinary’s Digital Asset Management Solution.

Today, I’m thrilled to announce the launch of Apps for Digital Asset Management and a Media Library Extension for the Chrome browser, which enables easy, flexible integration with all web-based applications in addition to making asset discovery more robust and accessible to all.

Read more
Scale and Automate Workflows With Modern Digital Asset Management Systems

With building, growing, and maintaining a strong digital presence being a top priority for all brands, high-quality visual content is paramount. In fact, consumers are 40 times more likely to share visual content on social networks than on other forums. Plus, a recent study from Wyzowl found that 84% of consumers made purchase decisions after watching a video, which explains why many brands are adding more and more visual media to their sites.

Read more