Why the obsession with quality?
The increasing complexity of today’s software products — combined with greater competitive pressures and skyrocketing costs of software breakdown — has pushed the need for testing to new heights. The quality (lack of bugs) of the software application is now a key factor in determining the success of the delivery.
The quality of a software product depends on the quality of the process that produced it. Similarly, the quality and effectiveness of software testing depends on the quality of the test processes followed. Because of this cascading effect, it is important to follow rules, guidelines, and processes (along with a sharp intuition!) for the effective testing of software.
Challenges of testing — is the application bug-free now?
Software quality may look like a simple concept on the surface, but it is not that straightforward in practice for a few reasons. For example:
Rapidly changing requirements.
Certain test scenarios that cannot be simulated.
Interaction with multiple out-of-scope applications.
For the above reasons (and challenges such as budget and time constraints), it may not be practical to fully test a product. Because of this reality, even after extensive testing of the application, there always remains a hunch that the application might fail at certain points. It is very important this intuition gets communicated. And this is where QA closure comes into the picture; a step which is important (yet underrated) in the software testing process. Let’s dive a little deeper into what it is and why it is important.
QA closure and the application health report
It is definitely a bad idea to just test as much as possible and then exit the process. After thoroughly testing the application, the QA team can provide the best review of the application before it is released. The best way to share this information is through formal documentation that can be shared with all of the stakeholders. But is it so important to share this information? The answer is yes, and here’s why:
It gives a final summary of the testing done.
Includes tests done, issues logged, types of testing done, OS/browser used, etc.
When we share this information with a client, it gives them a sense of security about the stability of the product they are receiving.
When we share this with our internal management, it can help them identify QA efforts in a summarized manner and analyze the quality of development.
It communicates clearly the areas of the application that were out of the scope of testing.
This helps in reminding the client, as well as management, that these might be areas of risk and might need extra attention post-release.
It communicates if a certain component or functionality does not seem robust.
This highlights components of the application which seem prone to failure in the future and explains why.
With this information, management can decide to move on or dig deeper into the issues and decide whether to resolve or ignore them.
It communicates if any in-scope testing is pending.
This means any testing that is pending due to timelines/budge/development issues.
It's important to raise this red flag to let management decide whether or not to go forward with the release of the product.
By providing the above visibility, we can inform stakeholders about the anticipated quality of an application. In addition, management is now able to take action on critical quality information before it turns into a crisis.
An ideal QA closure: what to include
The information provided in QA closure can vary widely based on the business situation or need. A basic QA closure must have 2 parts:
Part 1: QA reporting
This report includes information that is measurable. But once we start to think about what can be measured, it’s easy to be overwhelmed by the fact that we could measure almost everything. So we need to create priorities for measurement based on what measures are critical and will actually be of use once we have them.
The basic information that needs to go into every QA Closure report includes:
Types of testing done.
OS/browser used for testing.
Devices used for testing.
Test case counts — failed and passed.
Issues logged — opened and closed.
Issue density (e.g., issues logged, test cases executed).
The above information should be presented in percentage form, so that testing coverage can be inferred. Also QA Closure should have links to the data for the above subjects, as stakeholders would generally like to have a detailed view of a problem area if it exists.
Part 2: a review of the application by the QA team
This section of the QA closure document can include information like:
Risks and concerns as foreseen by QA.
Overall review of the application.
Feedback on the project processes, etc.
Although this is the section where the testing team can voice their opinion about the application, it is still best to be very specific and refrain from subjective opinion so that these risks and concerns can actually be identified and resolved.
In addition to testing, the QA team can further contribute to the success of the application by sharing the right data at the right time, by providing feedback, and/or raising any concerns in a timely manner.