Łukasz Makuch

If you've ever asked yourself the question "Why did they write these tests?" then this blog post is for you. Over the last decade, I've observed a multitude of reasons as to why IT professionals automate tests. Some of them are purely technical, and some are rather social. Some bring a lot of value to the company, while others benefit only individual engineers. Revealing them here will ruffle some feathers, but it can also serve as a nudge to abandon some practices while doubling down on others.

The 14 reasons to automate tests

1. Spotting regression as soon as possible

One of the most common reasons to automate tests is to spot regression. Once a system grows big, making sure everything still works as expected requires a lot of manual work. Hiring manual testers is also expensive, and even an entire team of manual testers may still need a significant amount of time to thoroughly test a big system. This is the reason why testing tickets pile up before releases. That's also why developers become hesitant to introduce changes that would pay off in the future but could introduce bugs the QA wouldn't find. Such decisions provide short-term stability and faster release cycles but lead to the accumulation of technical debt, which slows the development process down in the long run. Cheap and fast automated tests enable teams to release more often and stay away from technical debt. Also, the whole process feels much calmer without all that fear about breaking who knows what without even knowing about it.

2. Verifying properties, broadly speaking, not just some cases

Imagine we're creating an algorithm that should output a number between 0 and 1. If it produces a result beyond that range, it means there's an error. How do we go about testing this? An easy way to increase our confidence in this algorithm is to use property testing. While it's oftentimes not feasible to manually create thousands or millions of inputs, computers can do it just fine. It means that instead of writing a scenario like "Given 42 and 7, it returns 0.3", we can write a scenario like "Given 2 numbers, it returns a number between 0 and 1" and the computer will generate and run tons of concrete test cases to ensure that's true.

3. Testing what's hard to test manually

It's relatively straightforward to manually test happy path scenarios. However, testing becomes increasingly more challenging if we need to check if the system works as expected when something goes wrong. Think a dependency service being down or concurrent processes executing in an unusual order. Fortunately, in a scripted test environment we can have total control over all the moving pieces and it becomes easy to simulate all the edge cases.

4. Testing what nobody wants to test manually

Some things are worth keeping in check, but it's really mundane to manually enforce them. Examples of such things include the right letter capitalization or the right number of spaces. While a human can certainly look for white space violations, there are probably better ways for them to make good use of their time. Automating this aspect of product development frees people to do things that are more valuable for the business.

5. Meeting contractual obligations

Many kinds of tests, when done right, prove invaluable in increasing the confidence we can have in our products, which leads to faster time to market. That's why a requirement to have automated tests may be found in various contracts, for example, when a company is looking for investors. Such contractual obligations may even serve as the primary incentive for teams to embrace automated testing.

6. Social pressure

Contrary to what some may believe, engineers are social creatures too, and that makes them susceptible to social pressure. So even if an individual contributor doesn't find the practice of test automation beneficial, they may still be tempted to automate them in an environment that favours those who do. Being accepted by one's peers or getting that life-changing promotion is worth doing things that seem pointless.

7. Learning how to automate tests

To err is human, but humans also learn by doing. So even if we don't know how to effectively automate tests, it may still be a good idea to at least try to do it. There's a chance that after enough attempts we'll start seeing a return on investment.

8. Spotting security vulnerabilities before the bad actors do

Nowadays, it's rare for any big product to be built entirely from scratch. Most of the time, there are countless prefabricated components, such as libraries, compilers, and even operating systems. It means that even if the code of our product is perfectly secure, it may still be compromised if a security vulnerability is discovered in one of its underpinning technologies. That's why it's important to continuously monitor the codebase for known security issues. An automatic security check can be scheduled to run on weekends and during the night just fine. That way, we can react to threats faster. And because a human is alerted only if needed, it helps to keep the cost low.

9. Measuring performance

Modern multi-tenant web apps are no stangers to handling tens of thousands of requests per second. Let's say we're about to launch a new feature. How can we measure its performance, so that we have at least some idea about how much the infrastructure will cost? If you've ever tried clicking very fast to see how an app perfroms under heavy load, you know that it may be quite a challenge! That's where automated stress testing comes into play. Being able to repeadetly generate the same amount of traffic and measure various parameters such as CPU load and memory usage proves invaluable in making fact-driven decisions.

10. Seeing how different variants perform

Let's say there are two variants of the same thing in our product: variant A and variant B. Will our users like the variant A more than the variant B? It may be hard to predict. Instead of trying to predict it, we can deploy both variants, measure their key performance indicators, and automatically switch to the variant that performs better.

11. Making use of the byproducts of design to speed up the development process

While in many ways it's true that building software is like building a plane in the air, some products or pieces of thereof are still designed upfront. It may help us if we ask ourselves the question "So what is it supposed to do, again?". The outcome of answering it may be a set of precise preconditions, actions and expectations, akin to the components of an automated test suite. If these descriptions are so precise, then we're just one step away from automating the process of verifying them. And if we can easily shave off hours, if not days, of mundane manual labour, why not do that?

12. Recreating scenarios

Automated tests often consist of 3 phases, namely arranging, acting, and asserting. Even if we want to manually test some scenario, we may not be keen on arranging all the test data and navigating through the app to reach the testing point. In such situation, a hybrid approach may emerge, where the first phase - arranging - is automated, while the 2 remaining ones are done manually.

13. Fostering communication

Sometimes it's easier to talk about a product through the lens of its tests. A language like Gherkin encourages the formulation of human-readable yet specific and actionable examples. Even failed tests can help us communicate with others, especially when using frontend-testing-tools. In such tools, each failed test generates a before-and-after picture, visually highlighting the difference between the reference image and the current state of the app.

14. Supporting individuals with disabilities

There are many reasons why the selectors from Testing Library took the community by storm. One of them is the fact that they are accessible by default. For many people, it's enough for a rectangle to appear on the screen to be considered a button. However, others may benefit from that rectangle having a specific button role, properly attached label text, and support for keyboard-driven interactions. Although the selectors provided by today's state-of-the-art libraries still don't give us a guarantee that the app is fully inclusive, they're certainly a step in the right direction that can eliminate the most common barriers troubling people who rely on accessible web applications.

Summary

In an industry where new ways to develop software emerge more often than new ways to test that software, testing is a delicate topic. There's still a lot to learn and invent. I believe that what's even more important than learning particular tools is to understand the real outcomes of introducing some testing regimes, as well as being honest about our motivation.