What is Exploratory Testing? Benefits, Examples, How-To
Exploratory testing is widely used in Agile models, and it allows testers to go beyond the limits of scripted testing.
Think of scripted testing as taking a train. It's on rails. It will never deviate from that path. Exploratory testing is more like driving a car. Your general goal is to get from point A to B, but you can take any path and deviation that catches your eye along the way.
So What is Exploratory Testing?
Exploratory software testing is done on the fly and relies on the tester to think beyond the limits of the scripted tests. Exploratory testers design a test, execute it immediately, observe the results, and use results to design the next test.
Why Do Exploratory Testing
As one can guess, this approach can take more time than scripted tests, and it’s completed in addition to scripted testing — so it may be tempting to omit.
However, exploratory testing finds more defects.
Here are some statistics:
- On average, exploratory testing discovers 11% more overall defects than scripted testing.
- For defects that should be immediately obvious, such as a missing button in the UI, exploratory testing discovers 29% more vs. scripted.
- When it comes to “complex” bugs (bugs requiring three or more user actions to cause an error or failure), it jumps to 33% more defects found.
You find more defects with exploratory testing because you have more latitude to try different types of tests while using your past experience and knowledge of the product. Scripted testing, on the other hand, limits you to only the steps outlined in test cases, which in turn limits your ability to consider other test scenarios.
▶️ WATCH >>TURN BY TURN: A PRACTICAL GUIDE TO TEST CASE MANAGEMENT
Why Test Cases Don’t Always Cover Your Bases
Even if you had “perfect” test cases, exploratory testing would still find more defects over the course of a release for several reasons:
- You tend to find a good number of defects when “testing around” functional areas while verifying defects. Fixing an issue often breaks something else.
- If a defect exists and is not found while executing the initial test run (following the test cases steps), it is unlikely that the next tester running the same test will find the defect. However, exploratory testing in the same functional area may reveal the bug.
- Exploratory testing allows you to think outside the box and come up with use cases that might not be covered in a test case. For example, you might perform one test and then ask yourself, “What if I tried this? What if I didn’t do that?”
- Some defects, typically the hard ones to find, are dependent on a sequence of events. If you don’t have really deep test cases, you can miss finding defects that exploratory testing can find in a less structured, but longer, test session.
>> Also learn about regression testing and continuous testing.
Exploratory Testing Example
Because you are not bound by the test case steps, exploratory testing lets you find important defects faster. It allows you to cover more ground and focus on testing the “what ifs.”
For example, let’s say you’re assigned to test the “Edit Bookmark” functional area in a product.
There are two possible scenarios for your assignment:
1) Execute a test run
-OR-
2) Perform exploratory testing of the “Edit Bookmark” functional area.
In the first scenario, you would follow the steps outlined in the test case to verify that each step works, reporting any bugs that are found. But what if a critical bug — such as a crash, or data loss — doesn’t occur during these steps?
You won’t find it.
In the second scenario, you explore or “test around” editing bookmarks. At first, you might perform the same steps as at the beginning of the test case. As testing progresses, however, you might ask, “What happens when I click Edit, delete the name, and try to save the bookmark with a blank field?” Boom! The application crashes and deletes all of the user data. You just found the most important bug by exploring something that wasn’t a step in the test case.
How to Structure Exploratory Testing
Just because this approach is done on the fly doesn’t mean it isn’t structured. Exploratory test management needs to ensure that the following happens:
- A clear mission of the test is established.
- Thorough notes are taken of what needs to be tested, why, and the assessed quality of the application.
- Issues and questions raised during testing are tracked.
- Two testers are appropriately assigned based on their experience.
This is accomplished through 5 phases, or session based test management (SBTM Cycle):
1. Classify the bugs.
- Categorize the kinds of problems most commonly found in past projects.
- Seek the root cause of these problems.
- Identify the risks and develop ideas to test the application.
2. Create a test charter.
- Identify what to test, how it can be tested, and what factors to consider.
- Describe the starting point of testing.
- Define how users will utilize the system.
3. Create a time box.
- Two testers work together for at least 90 minutes.
- Interruptions do not occur during the 90 minutes.
- An extension or reduction of 45 minutes is acceptable.
- Testers react to the response of the application and prepare for the correct outcome.
4. Review the results.
- Assess the defects.
- Learn from the test.
- Analyze coverage areas.
5. Debrief.
- Compile results.
- Compare results to the character.
- Check for needed additional testing.
LEARN TIPS FOR WRITING TEST CASES >>
Maintaining a Tight Schedule With Added Testing
Combining exploratory testing with mandated scripted testing can keep your test effort strong and reduce the risk of defects in the release.
Are you finding this type of testing to be worthwhile, but have limited resources? You may be able to free up resources elsewhere in your process.
For example, Helix TCM can make it easier to write, execute, and track your scripted tests. It pairs with other tools, like Jira, allowing your team to keep tools they already know.
And companies with more accurate and efficient processes ship products faster.
See if you can make your current process more efficient to strengthen your testing as a whole.
