{"id":22358,"date":"2021-05-25T13:59:48","date_gmt":"2021-05-25T13:59:48","guid":{"rendered":"http:\/\/mythbusters_part_2_automated_ui_tests_are_slow"},"modified":"2021-05-25T13:59:48","modified_gmt":"2021-05-25T13:59:48","slug":"mythbusters_part_2_automated_ui_tests_are_slow","status":"publish","type":"post","link":"https:\/\/cloudinary.com\/blog\/mythbusters_part_2_automated_ui_tests_are_slow","title":{"rendered":"MythBusters, Part 2: Automated UI Tests Are Slow"},"content":{"rendered":"

Part 1 of MythBusters<\/a> busted the myth of unstable tests. Part 2 here smashes another myth: \u201cUI tests are slow.\u201d Far be it from me to dispute that UI tests are slower than unit or integration tests, but let me show you the best practices that speed up UI tests.<\/p>\n

Let\u2019s first consider the steps you\u2019d take for a typical, simple test:<\/p>\n

    \n
  1. Log in to the system.<\/li>\n
  2. Prepare the data for the test.<\/li>\n
  3. Validate that the used data appears as required.<\/li>\n<\/ol>\n

    To execute a slow test with Selenium, you\u2019d take these three steps:<\/p>\n

      \n
    1. To log in, you must navigate to the home page, click the login icon, and type in your credentials.<\/li>\n
    2. To prepare the data for the test, you must create the data in the UI.<\/li>\n
    3. To validate the data, you must take a few steps.<\/li>\n<\/ol>\n

      Depending on the speed of the network and database usage, such a typical test at Cloudinary takes 20-30 seconds.<\/p>\n

      SETUP TEST DATA VIA API AND NOT UI<\/strong><\/p>\n

      However, that same test in our test suite takes only 10 seconds, a 50-percent reduction in time spent<\/strong>. That might seem insignificant, but if there are thousands of tests, those seconds add up, resulting in many expensive minutes. Here are the steps we take to accelerate the process:<\/p>\n

        \n
      1. For logins, we use an authentication API<\/a> so that we are automatically logged in to the system after navigating to the home page.<\/li>\n
      2. For data preparation, we also use an API. For example, in our automation framework, we upload media assets with SDK<\/a> calls.<\/li>\n
      3. For validation, we follow the same steps as those in the \u201clong\u201d test.<\/li>\n<\/ol>\n

        As to the parallelism of test execution, many frameworks offer parallel execution out of the box. To prepare for that, we do the following:<\/p>\n

          \n
        1. \n

          Write independent tests<\/strong>. Our tests must be independent. Dependent tests might run in parallel, but they would fail due to the dependencies. For example, in case of parallel execution, Test 2 might run before Test 1, but if Test 1 depends on Test 2, Test 1 will fail.<\/p>\n<\/li>\n

        2. \n

          Use parallel test runners<\/strong>. We must prepare solutions for continuous integration<\/a> (CI) by leveraging free tools like Selenium Grid<\/a> and Selenoid<\/a>, or paid solutions. We can also run tests in parallel with the framework without any third-party software.<\/p>\n<\/li>\n

        3. \n

          Aim for cost-effectiveness<\/strong>. Because parallel execution requires more CPU and memory, we must boost the performance of the executing machines, i.e., find the sweet spot between lab size and test execution time.<\/p>\n

          At Cloudinary, we use five instances of AWS and execute 10 parallel test runners.<\/p>\n<\/li>\n

        4. \n

          Optimize the test size<\/strong>. Another important factor that affects time performance is the test size. Tests should be short and should check only what is required. In manual testing, we can check many details along the way, e.g., the login process, links, colors, fonts, etc. However, in an automated test, we need to check only one thing.<\/p>\n

          Also, remember that not every manual test needs to be automated. In case of bugs, if your automation framework is built correctly, you will see only the failures that relate to those bugs, which is a big help for debugging.<\/p>\n<\/li>\n<\/ol>\n

          \"automated\nAt Cloudinary, we follow all the rules described above for over 1,000 automated tests with a total execution time of about 40 minutes. We don\u2019t stop there, however, and will continue to look for ways to save even more time in automated tests.<\/p>\n<\/div>","protected":false},"excerpt":{"rendered":"","protected":false},"author":41,"featured_media":22359,"comment_status":"","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"_cloudinary_featured_overwrite":false,"footnotes":""},"categories":[1],"tags":[],"class_list":["post-22358","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-uncategorized"],"acf":[],"yoast_head":"\nHow to Speed Up Automated UI Tests<\/title>\n<meta name=\"description\" content=\"To speed up automated tests, Cloudinary authenticates logins and prepares data with APIs besides adopting best practices like writing independent tests.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/cloudinary.com\/blog\/mythbusters_part_2_automated_ui_tests_are_slow\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"MythBusters, Part 2: Automated UI Tests Are Slow\" \/>\n<meta property=\"og:description\" content=\"To speed up automated tests, Cloudinary authenticates logins and prepares data with APIs besides adopting best practices like writing independent tests.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/cloudinary.com\/blog\/mythbusters_part_2_automated_ui_tests_are_slow\" \/>\n<meta property=\"og:site_name\" content=\"Cloudinary Blog\" \/>\n<meta property=\"article:published_time\" content=\"2021-05-25T13:59:48+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/res.cloudinary.com\/cloudinary-marketing\/images\/f_auto,q_auto\/v1649719121\/Web_Assets\/blog\/mythbusters_part-2_223590e71a\/mythbusters_part-2_223590e71a.png?_i=AA\" \/>\n\t<meta property=\"og:image:width\" content=\"1400\" \/>\n\t<meta property=\"og:image:height\" content=\"770\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"NewsArticle\",\"@id\":\"https:\/\/cloudinary.com\/blog\/mythbusters_part_2_automated_ui_tests_are_slow#article\",\"isPartOf\":{\"@id\":\"https:\/\/cloudinary.com\/blog\/mythbusters_part_2_automated_ui_tests_are_slow\"},\"author\":{\"name\":\"\",\"@id\":\"\"},\"headline\":\"MythBusters, Part 2: Automated UI Tests Are Slow\",\"datePublished\":\"2021-05-25T13:59:48+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/cloudinary.com\/blog\/mythbusters_part_2_automated_ui_tests_are_slow\"},\"wordCount\":7,\"publisher\":{\"@id\":\"https:\/\/cloudinary.com\/blog\/#organization\"},\"image\":{\"@id\":\"https:\/\/cloudinary.com\/blog\/mythbusters_part_2_automated_ui_tests_are_slow#primaryimage\"},\"thumbnailUrl\":\"https:\/\/res.cloudinary.com\/cloudinary-marketing\/images\/f_auto,q_auto\/v1649719121\/Web_Assets\/blog\/mythbusters_part-2_223590e71a\/mythbusters_part-2_223590e71a.png?_i=AA\",\"inLanguage\":\"en-US\",\"copyrightYear\":\"2021\",\"copyrightHolder\":{\"@id\":\"https:\/\/cloudinary.com\/#organization\"}},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/cloudinary.com\/blog\/mythbusters_part_2_automated_ui_tests_are_slow\",\"url\":\"https:\/\/cloudinary.com\/blog\/mythbusters_part_2_automated_ui_tests_are_slow\",\"name\":\"How to Speed Up Automated UI Tests\",\"isPartOf\":{\"@id\":\"https:\/\/cloudinary.com\/blog\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/cloudinary.com\/blog\/mythbusters_part_2_automated_ui_tests_are_slow#primaryimage\"},\"image\":{\"@id\":\"https:\/\/cloudinary.com\/blog\/mythbusters_part_2_automated_ui_tests_are_slow#primaryimage\"},\"thumbnailUrl\":\"https:\/\/res.cloudinary.com\/cloudinary-marketing\/images\/f_auto,q_auto\/v1649719121\/Web_Assets\/blog\/mythbusters_part-2_223590e71a\/mythbusters_part-2_223590e71a.png?_i=AA\",\"datePublished\":\"2021-05-25T13:59:48+00:00\",\"description\":\"To speed up automated tests, Cloudinary authenticates logins and prepares data with APIs besides adopting best practices like writing independent tests.\",\"breadcrumb\":{\"@id\":\"https:\/\/cloudinary.com\/blog\/mythbusters_part_2_automated_ui_tests_are_slow#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/cloudinary.com\/blog\/mythbusters_part_2_automated_ui_tests_are_slow\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/cloudinary.com\/blog\/mythbusters_part_2_automated_ui_tests_are_slow#primaryimage\",\"url\":\"https:\/\/res.cloudinary.com\/cloudinary-marketing\/images\/f_auto,q_auto\/v1649719121\/Web_Assets\/blog\/mythbusters_part-2_223590e71a\/mythbusters_part-2_223590e71a.png?_i=AA\",\"contentUrl\":\"https:\/\/res.cloudinary.com\/cloudinary-marketing\/images\/f_auto,q_auto\/v1649719121\/Web_Assets\/blog\/mythbusters_part-2_223590e71a\/mythbusters_part-2_223590e71a.png?_i=AA\",\"width\":1400,\"height\":770},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/cloudinary.com\/blog\/mythbusters_part_2_automated_ui_tests_are_slow#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/cloudinary.com\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"MythBusters, Part 2: Automated UI Tests Are Slow\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/cloudinary.com\/blog\/#website\",\"url\":\"https:\/\/cloudinary.com\/blog\/\",\"name\":\"Cloudinary Blog\",\"description\":\"\",\"publisher\":{\"@id\":\"https:\/\/cloudinary.com\/blog\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/cloudinary.com\/blog\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/cloudinary.com\/blog\/#organization\",\"name\":\"Cloudinary Blog\",\"url\":\"https:\/\/cloudinary.com\/blog\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/cloudinary.com\/blog\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/res.cloudinary.com\/cloudinary-marketing\/images\/f_auto,q_auto\/v1649718331\/Web_Assets\/blog\/cloudinary_logo_for_white_bg_1937437aa7_19374666c7_193742f877\/cloudinary_logo_for_white_bg_1937437aa7_19374666c7_193742f877.png?_i=AA\",\"contentUrl\":\"https:\/\/res.cloudinary.com\/cloudinary-marketing\/images\/f_auto,q_auto\/v1649718331\/Web_Assets\/blog\/cloudinary_logo_for_white_bg_1937437aa7_19374666c7_193742f877\/cloudinary_logo_for_white_bg_1937437aa7_19374666c7_193742f877.png?_i=AA\",\"width\":312,\"height\":60,\"caption\":\"Cloudinary Blog\"},\"image\":{\"@id\":\"https:\/\/cloudinary.com\/blog\/#\/schema\/logo\/image\/\"}},{\"@type\":\"Person\",\"@id\":\"\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"How to Speed Up Automated UI Tests","description":"To speed up automated tests, Cloudinary authenticates logins and prepares data with APIs besides adopting best practices like writing independent tests.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/cloudinary.com\/blog\/mythbusters_part_2_automated_ui_tests_are_slow","og_locale":"en_US","og_type":"article","og_title":"MythBusters, Part 2: Automated UI Tests Are Slow","og_description":"To speed up automated tests, Cloudinary authenticates logins and prepares data with APIs besides adopting best practices like writing independent tests.","og_url":"https:\/\/cloudinary.com\/blog\/mythbusters_part_2_automated_ui_tests_are_slow","og_site_name":"Cloudinary Blog","article_published_time":"2021-05-25T13:59:48+00:00","og_image":[{"width":1400,"height":770,"url":"https:\/\/res.cloudinary.com\/cloudinary-marketing\/images\/f_auto,q_auto\/v1649719121\/Web_Assets\/blog\/mythbusters_part-2_223590e71a\/mythbusters_part-2_223590e71a.png?_i=AA","type":"image\/png"}],"twitter_card":"summary_large_image","schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"NewsArticle","@id":"https:\/\/cloudinary.com\/blog\/mythbusters_part_2_automated_ui_tests_are_slow#article","isPartOf":{"@id":"https:\/\/cloudinary.com\/blog\/mythbusters_part_2_automated_ui_tests_are_slow"},"author":{"name":"","@id":""},"headline":"MythBusters, Part 2: Automated UI Tests Are Slow","datePublished":"2021-05-25T13:59:48+00:00","mainEntityOfPage":{"@id":"https:\/\/cloudinary.com\/blog\/mythbusters_part_2_automated_ui_tests_are_slow"},"wordCount":7,"publisher":{"@id":"https:\/\/cloudinary.com\/blog\/#organization"},"image":{"@id":"https:\/\/cloudinary.com\/blog\/mythbusters_part_2_automated_ui_tests_are_slow#primaryimage"},"thumbnailUrl":"https:\/\/res.cloudinary.com\/cloudinary-marketing\/images\/f_auto,q_auto\/v1649719121\/Web_Assets\/blog\/mythbusters_part-2_223590e71a\/mythbusters_part-2_223590e71a.png?_i=AA","inLanguage":"en-US","copyrightYear":"2021","copyrightHolder":{"@id":"https:\/\/cloudinary.com\/#organization"}},{"@type":"WebPage","@id":"https:\/\/cloudinary.com\/blog\/mythbusters_part_2_automated_ui_tests_are_slow","url":"https:\/\/cloudinary.com\/blog\/mythbusters_part_2_automated_ui_tests_are_slow","name":"How to Speed Up Automated UI Tests","isPartOf":{"@id":"https:\/\/cloudinary.com\/blog\/#website"},"primaryImageOfPage":{"@id":"https:\/\/cloudinary.com\/blog\/mythbusters_part_2_automated_ui_tests_are_slow#primaryimage"},"image":{"@id":"https:\/\/cloudinary.com\/blog\/mythbusters_part_2_automated_ui_tests_are_slow#primaryimage"},"thumbnailUrl":"https:\/\/res.cloudinary.com\/cloudinary-marketing\/images\/f_auto,q_auto\/v1649719121\/Web_Assets\/blog\/mythbusters_part-2_223590e71a\/mythbusters_part-2_223590e71a.png?_i=AA","datePublished":"2021-05-25T13:59:48+00:00","description":"To speed up automated tests, Cloudinary authenticates logins and prepares data with APIs besides adopting best practices like writing independent tests.","breadcrumb":{"@id":"https:\/\/cloudinary.com\/blog\/mythbusters_part_2_automated_ui_tests_are_slow#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/cloudinary.com\/blog\/mythbusters_part_2_automated_ui_tests_are_slow"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/cloudinary.com\/blog\/mythbusters_part_2_automated_ui_tests_are_slow#primaryimage","url":"https:\/\/res.cloudinary.com\/cloudinary-marketing\/images\/f_auto,q_auto\/v1649719121\/Web_Assets\/blog\/mythbusters_part-2_223590e71a\/mythbusters_part-2_223590e71a.png?_i=AA","contentUrl":"https:\/\/res.cloudinary.com\/cloudinary-marketing\/images\/f_auto,q_auto\/v1649719121\/Web_Assets\/blog\/mythbusters_part-2_223590e71a\/mythbusters_part-2_223590e71a.png?_i=AA","width":1400,"height":770},{"@type":"BreadcrumbList","@id":"https:\/\/cloudinary.com\/blog\/mythbusters_part_2_automated_ui_tests_are_slow#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/cloudinary.com\/blog\/"},{"@type":"ListItem","position":2,"name":"MythBusters, Part 2: Automated UI Tests Are Slow"}]},{"@type":"WebSite","@id":"https:\/\/cloudinary.com\/blog\/#website","url":"https:\/\/cloudinary.com\/blog\/","name":"Cloudinary Blog","description":"","publisher":{"@id":"https:\/\/cloudinary.com\/blog\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/cloudinary.com\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/cloudinary.com\/blog\/#organization","name":"Cloudinary Blog","url":"https:\/\/cloudinary.com\/blog\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/cloudinary.com\/blog\/#\/schema\/logo\/image\/","url":"https:\/\/res.cloudinary.com\/cloudinary-marketing\/images\/f_auto,q_auto\/v1649718331\/Web_Assets\/blog\/cloudinary_logo_for_white_bg_1937437aa7_19374666c7_193742f877\/cloudinary_logo_for_white_bg_1937437aa7_19374666c7_193742f877.png?_i=AA","contentUrl":"https:\/\/res.cloudinary.com\/cloudinary-marketing\/images\/f_auto,q_auto\/v1649718331\/Web_Assets\/blog\/cloudinary_logo_for_white_bg_1937437aa7_19374666c7_193742f877\/cloudinary_logo_for_white_bg_1937437aa7_19374666c7_193742f877.png?_i=AA","width":312,"height":60,"caption":"Cloudinary Blog"},"image":{"@id":"https:\/\/cloudinary.com\/blog\/#\/schema\/logo\/image\/"}},{"@type":"Person","@id":""}]}},"jetpack_featured_media_url":"https:\/\/res.cloudinary.com\/cloudinary-marketing\/images\/f_auto,q_auto\/v1649719121\/Web_Assets\/blog\/mythbusters_part-2_223590e71a\/mythbusters_part-2_223590e71a.png?_i=AA","_links":{"self":[{"href":"https:\/\/cloudinary.com\/blog\/wp-json\/wp\/v2\/posts\/22358","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/cloudinary.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/cloudinary.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/cloudinary.com\/blog\/wp-json\/wp\/v2\/users\/41"}],"replies":[{"embeddable":true,"href":"https:\/\/cloudinary.com\/blog\/wp-json\/wp\/v2\/comments?post=22358"}],"version-history":[{"count":0,"href":"https:\/\/cloudinary.com\/blog\/wp-json\/wp\/v2\/posts\/22358\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/cloudinary.com\/blog\/wp-json\/wp\/v2\/media\/22359"}],"wp:attachment":[{"href":"https:\/\/cloudinary.com\/blog\/wp-json\/wp\/v2\/media?parent=22358"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/cloudinary.com\/blog\/wp-json\/wp\/v2\/categories?post=22358"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/cloudinary.com\/blog\/wp-json\/wp\/v2\/tags?post=22358"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}