Cloudinary Blog

MythBusters, Part 2: Automated UI Tests Are Slow

Alexey Kagansky
By Alexey Kagansky
How to Speed Up Automated UI Tests

Part 1 of MythBusters busted the myth of unstable tests. Part 2 here smashes another myth: “UI tests are slow.” Far be it from me to dispute that UI tests are slower than unit or integration tests, but let me show you the best practices that speed up UI tests.

Let's first consider the steps you’d take for a typical, simple test:

  1. Log in to the system.
  2. Prepare the data for the test.
  3. Validate that the used data appears as required.

To execute a slow test with Selenium, you’d take these three steps:

  1. To log in, you must navigate to the home page, click the login icon, and type in your credentials.
  2. To prepare the data for the test, you must create the data in the UI.
  3. To validate the data, you must take a few steps.

Depending on the speed of the network and database usage, such a typical test at Cloudinary takes 20-30 seconds.

SETUP TEST DATA VIA API AND NOT UI

However, that same test in our test suite takes only 10 seconds, a 50-percent reduction in time spent. That might seem insignificant, but if there are thousands of tests, those seconds add up, resulting in many expensive minutes. Here are the steps we take to accelerate the process:

  1. For logins, we use an authentication API so that we are automatically logged in to the system after navigating to the home page.
  2. For data preparation, we also use an API. For example, in our automation framework, we upload media assets with SDK calls.
  3. For validation, we follow the same steps as those in the “long” test.

As to the parallelism of test execution, many frameworks offer parallel execution out of the box. To prepare for that, we do the following:

  1. Write independent tests. Our tests must be independent. Dependent tests might run in parallel, but they would fail due to the dependencies. For example, in case of parallel execution, Test 2 might run before Test 1, but if Test 1 depends on Test 2, Test 1 will fail.

  2. Use parallel test runners. We must prepare solutions for continuous integration (CI) by leveraging free tools like Selenium Grid and Selenoid, or paid solutions. We can also run tests in parallel with the framework without any third-party software.

  3. Aim for cost-effectiveness. Because parallel execution requires more CPU and memory, we must boost the performance of the executing machines, i.e., find the sweet spot between lab size and test execution time.

    At Cloudinary, we use five instances of AWS and execute 10 parallel test runners.

  4. Optimize the test size. Another important factor that affects time performance is the test size. Tests should be short and should check only what is required. In manual testing, we can check many details along the way, e.g., the login process, links, colors, fonts, etc. However, in an automated test, we need to check only one thing.

    Also, remember that not every manual test needs to be automated. In case of bugs, if your automation framework is built correctly, you will see only the failures that relate to those bugs, which is a big help for debugging.

automated tests
At Cloudinary, we follow all the rules described above for over 1,000 automated tests with a total execution time of about 40 minutes. We don't stop there, however, and will continue to look for ways to save even more time in automated tests.

Recent Blog Posts

Automate the Staging Process of Videos for Social Media

Rich and engaging media helps build customer engagement and trust but can be time consuming to stage. Developers save a tremendous amount of time by preparing videos for social media with Cloudinary. That’s because Cloudinary’s interface, widgets, and application programming interface (API) transform raw media into polished content, optimizing footage and enabling effortless customization and publishing.

Read more

Top Five Web-Video Formats of 2021

By William Imoh
The Five Most Popular Web-Video Formats and Streaming Protocols

Over the past 15 years, the video industry has undergone a significant change in video formats on the web. In particular, in the early 2010s, the 3GP format, which the 3rd Generation Partnership Project (3GPP) created for 3G-enabled mobile devices, went nearly extinct. The advancement of mobile devices and cellular networks has brought about the need for pioneers to build better formats for a faster user experience.

Read more
Cloudinary Introduces Integration With SAP Commerce Cloud

We’re excited to announce Cloudinary’s integration with SAP Commerce Cloud, through which the latter’s customers can significantly boost the visual media experience on their website or app.

SAP Commerce Cloud powers some of the largest e-commerce sites (B2C, B2B, and B2B2C businesses), complete with building blocks like storefront design and order management. Reinforced with Cloudinary’s laser-sharp focus on optimizing, managing, and delivering images and videos, the new extension will enable SAP Commerce Cloud customers to create unique and engaging visual experiences effortlessly.

Read more
Personalizing Video Email for Marketing Campaigns With Cloudinary

As critical as it is to engage with shoppers in order to succeed in e-commerce, old-style, boring emails are far from being effective. In fact, they tend to be annoying because no one likes to read formulaic, generic messages that are sent en masse. For better results, rethink your email marketing campaigns and try out creative strategies.

Read more
Muted Videos and Subtitles

The bane of our existence is the lack of efficient ways for tackling the plethora of recurring tasks in our lives. One of those tasks is surfing the internet. We consume a lot of web content daily, of which a large percentage are images and videos. We’re constantly quickly scrolling through 30-second videos or checking out pictures of cute items we’d like to buy in our free time.

Read more

Building a Roommate-Matching App With Cloudinary and Jamstack

By Marcelo Ricardo de Oliveira
Building a Roommate-Matching App With Cloudinary and Jamstack

Roommate matching can be a pain—especially during the COVID pandemic when people don't want to meet in person. Matching apps like Flatmates, Roomster, and roommates.com are helpful, and if you're in the roommate-matching space, you know that great video is essential for those seeking roommates. Fortunately, Cloudinary can help.

Read more