Should I automate every test?

One of the most common questions in QA. The answer is no, and precisely because we're fans of automation, we know why.

It's the question that comes up on every QA team, usually during sprint planning, usually with a sigh attached. And the honest answer surprises people: no, you shouldn't automate every test.

That sounds strange coming from a company that lives off test automation. But it's precisely because we work with it every day that we know where the line sits. A team that tries to automate everything ends up with a suite nobody maintains.

What automation is genuinely good at

Automation is brilliant at the repeatable. A regression test that follows the same path every sprint, takes the same steps, expects the same outcome: let a machine do that. It doesn't get tired, doesn't skip a step, and doesn't complain that this is the two-hundredth run.

• Repetitive regression tests that need to run every release
• Performance and load tests
• API validations at scale
• Critical flows that simply must never break

What a script can't do

This is where it gets interesting. There's a category of test work where automation falls structurally short, and it has nothing to do with technical limitations. It's about judgment.

A script checks whether the button works. It doesn't notice that the button sits in a strange place. It checks whether the form submits. It doesn't notice that the error message is unreadable to a user in a hurry.

• Human judgment: assessing whether a new feature feels logical and usable
• Intuition: that sense that something's off, before you can say what
• Exploratory testing: discovering what you didn't know you had to test
• User experience: does it work, versus does it feel right

The wrong question

"Should I automate every test?" is actually the wrong question. The better question is: which test belongs where?

In practice we see that teams who make this trade-off deliberately build better suites. Not bigger, better. They automate what runs often and stays stable, and they keep human attention for where it pays off most: the new feature, the unexpected scenario, the question of whether this was even a good idea.

That's not a compromise. That's the point.

How to apply it in practice

Ask two questions for every test. How often does it run, and how stable is what it tests? Does it run every sprint and does the underlying feature rarely change: automate. Does it run once for a new feature whose design is still very much in flux: keep it manual, and only automate once it's stable.

Also look at the maintenance cost. An automated test that constantly needs adjusting often delivers less than it costs.

The principle is simple. Making the right call in practice often proves harder.