4 Automated Testing Myths BUSTED
When to Use Automated Testing
Automated testing helps you test more and test efficiently.
But there are some myths about automated testing — and when to use it.
So, let's debunk those myths.
Myth #1: Automated Testing Can Replace All Our Other Testing
Automated testing won’t replace all your other types of software testing. Scripts won’t do all the looking you need.
That's because automated testing is really “automated checking.” The machine is performing exactly the same steps exactly the same way. And it's only “checking” exactly what you told it to check.
So, you’ll still need a human to look at the results.
For example...
Let’s say you’re checking a form used to set up a user’s new account. You write a test script that will populate all form fields with sample data, submit the form, and check to see if a new account is created from the sample data. The script runs and tells you everything is working.
Eventually someone manually tests the form. He or she enters similar sample data into the form, submits it, and a new account is created — so far, so good. However, the background image is broken. And automated testing didn't catch it.
Myth #2: Automated Testing Will Automate Everything
Automated test scripts are not endlessly reusable. Introducing something simple can really mess up the automated test.
For example...
You introduce an intermediate dialog. A human tester can easily deal with a new dialog. But the script must be modified to deal with the new condition.
This is easy if the dialog shows up all the time. But what if the dialog only appears in edge cases?
You'd need to rewrite each one of your scripts to respond to the change. And changes to the user interface, process, or rules can sometimes require a ton of rework on the scripts.
Remember the time you would have spent testing the changes? You'll spend it analyzing what scripts to change and how to change them.
And this gets even more difficult if you are testing a client/server product (and you want to add in multi-user testing). The scripts running each “agent” will need to be able to communicate changes to each other.
Myth #3: Anybody Can Write a Script
Automation solutions provide tools that make automating tests easier. But they can’t do everything for you. And they can’t do everything for someone with no testing or development experience.
For example...
Even a simple test will take some expertise to automate. More sophisticated tests will be increasingly difficult. There will always be a kinks to work out that may be beyond the reach of an inexperienced tester.
The more engineering acumen you have, the better you will be at writing:
- Modular scripts
- Data-driven scripts
- Scripts that build their own model to check
These scripts do more than just push the client around, in other words. You’d be disappointed if a manual tester didn't carefully look at the results generated from their input. And you should be disappointed in an automated script that ignores the results, too.
Myth #4: Just Run Your Scripts for Each Build!
Wouldn’t it be great if it were that easy?
In reality, now you need to start analyzing run results. And you need to maintain the scripts to keep up with system under test (SUT) changes.
The more scripts you have, the more things you need to look after.
For example...
You could just keep running the scripts against new builds if the SUT doesn’t change much. But if that’s the case, you probably don’t need to test the SUT as often. So, you won’t reap much of a benefit from automating the tests.
Continually testing something that hasn't changed does not make much sense — manually or automated.
But if the user interface is changing constantly, you’ll have a lot of script maintenance to do.
In that case, perhaps testing it manually would take less time than it would to maintain the scripts for automated testing.
The sweet spot of automated testing is often when the UI is stable, but the underlying code still changes in small and subtle ways.
What Test Cases Can Be Automated?
Find out which tests you can automate — and which ones are better left manual.