Exploratory testing can be seen as traveling on foot instead of taking the bus, offering the freedom to deviate from a specific route. As the Agile environment has efficient principles that allow quick responses to changes and the ability to deal with uncertainty, exploratory testing may seem like a perfect match for such projects. However, this is only partially true.
In reality, diverse reasons impede its usage in Agile projects, and companies should take them into account before designing QA strategies. In this article, I’ll focus on these factors and share insights into the most suitable cases for applying exploratory testing.
The Ins and Outs of Exploratory Testing
In traditional testing, QA teams rely on a structured, step-by-step approach. This means they, in advance, must compile user journeys that lay a foundation for test cases, regularly trace coverage with appropriate metrics, and can easily reproduce captured defects. Deviations from the scripts aren’t welcome, feedback on software quality isn’t so fast, and alterations to requirements slow down the process even more.
Additionally, ad hoc testing, being random, intuitive, and informal, involves learning a tested system to be aware of its specifics and execute further testing without a test plan. It isn’t a mandatory part of the testing process and mainly focuses on negative scenarios.
Exploratory testing, on the other hand, is an on-the-fly activity of discovering an application to identify and document software issues. Product learning, test design, and execution happen concurrently, emphasizing an unstructured nature of testing. It provides rapid feedback as it helps detect defects quickly and works well when requirements are changing, or the QA teams are pressed for time. That said, it’s important to note that it doesn’t ensure complete test coverage.
Best Exploratory Testing Use Cases in Agile
Agile software development assumes collaboration, high flexibility, quick response to changes, and short sprints (up to 2 weeks). QA teams can apply exploratory testing in 6 cases:
- Learning the system under test. Let’s assume that the QA team joined the project upon the completion of several sprints. Exploratory testing helps to quickly gain sufficient knowledge of product operation, obtain the first documented results, and build the ground for a test model for further script-based verifications.
- Approaching high-pressure deadlines. Supplementing acceptance testing with exploratory checks is a win-win approach, as it increases testing coverage and quickly assesses system quality.
- Lack of fresh perspective. Sometimes there’s a need to verify software soundness without following developed documentation and think outside the box to discover novel, not-so-obvious use cases. For instance, if you have some free time, you may ask project newcomers to perform an assessment with exploratory testing. While using unconventional thinking, they may ask additional questions on software specifics and detect undiscovered issues, thus enhancing test coverage.
- Limited budget. Even in cases of financial constraint, organizations can focus on software quality by hiring an exploratory testing expert who will investigate the application from an end-user perspective rather than spending time on creating test documentation, reporting, and other activities.
- An intricate system with multiple branches in logic and an endless variety of data sets. When building a test model for such solutions, different test design techniques are required and exploratory testing can cope with this task faster and better.
- Minimal requirements. Often in pursuit of a rapid increase in functionality, less time is spent on detailed work with requirements, which leads to high-level or even vague user stories. In this case, analyzing the interaction between the old and new software functionalities falls on the QA team’s shoulders. Exploratory testing performed by seasoned team members helps to quickly delve into these specifics, provide fast feedback for the developers, and write meaningful test cases for further usage by less experienced QA specialists.
Exploratory testing is well-suited for these scenarios, providing QA teams with a versatile approach to ensuring quality.
Flip Side of Exploratory Testing in Agile
Despite its advantages, there are situations where exploratory testing in Agile software development may not be the best choice. Consider the following key points:
Reason 1. Inexperienced QA team
Successful exploratory testing requires creative and experienced QA engineers familiar with similar software solutions, business logic, and technologies. This knowledge helps them identify defect-prone areas and efficiently decide which features to test based on the progress made.
Reason 2. Shortage of exploratory testing expertise
QA engineers should have a solid background in exploratory testing, its techniques, and best practices, as well as an understanding of when to apply freestyle and session-based approaches to maximize QA efficiency. A lack of knowledge will likely lead to poor testing results.
Reason 3. System criticality
When we talk about complicated IT products such as banking solutions or ERP software, the highest priority is to delve deep into all the aspects of their sophisticated business logic and requirements. It’s also vital to understand software architecture to detect issues and ensure continuous, flawless operation of mission-critical functionality.
Therefore, exploratory testing alone isn’t enough to test the nuances of complex software. Instead, one should utilize diverse test design techniques to ensure maximum test coverage.
Reason 4. A poor basis for introducing test automation
If you need to accelerate testing time and expand test coverage by introducing test automation, exploratory testing of any type isn’t fit for purpose. To set up automated workflows and obtain high ROI, you need to write test cases with structured test steps, preconditions, as well as expected and actual results.
Reason 5. A limited ability to learn from mistakes
Picture this: your team has performed exploratory testing, and the product has gone live, suddenly a critical defect occurs in the production environment negatively affecting stable software operation and worse, impeding profit.
After fixing the issue, the first thing you need to do is to analyze the root cause during a retrospective. This analysis allows you to adjust your test strategy so that this incident never happens again. Yet, when sticking to the traditional freestyle approach of exploratory testing, test run results may not be fully documented, complicating the process of learning from your mistakes and resulting in increased business risks.
If you still opt for exploratory testing, ensure you use a session-based approach. This involves writing a test charter with test objectives to guide the process and a test session sheet to document the steps and findings in order to make QA more structured.
Reason 6. Overlapping tests
In freestyle exploratory testing, the lack of formalized test documentation may result in duplicated work when the QA teams check the same scenarios more than once. It adversely impacts time to market and overall project efficiency. Part of the functionality can even be left untested. So, if you still consider exploratory testing a fit-for-purpose solution, choose its session-based variant. It requires identifying the testing scope for each team member in advance to avoid duplication of work.
Reason 7. Compliance-based testing
If there’s a need to ensure that the software meets international standards such as WCAG 2.2 or OWASP, the QA team must strictly follow a set checklist. FDA, HIPAA, and other healthcare regulations require even greater attention as each step should be documented to ensure the IT product provides secure storage and sharing of sensitive ePHI and doesn’t pose a threat to patients’ lives. Exploratory testing doesn’t align with these activities.
The Takeaway
The primary objectives of QA and testing activities in general are to ensure a sound, high-performance software operation that helps businesses attain set goals and for end users to obtain an intuitive, user-friendly IT product.
Exploratory testing, in turn, helps review the quality of a product from a user perspective, think outside the box, and find unexpected ways of uncovering concealed issues. Without a doubt, it’s an effective addition to a testing strategy but not the first necessity—especially for Agile projects. So, plan smartly and pick what’s best for your business.