The True Purpose of Testing
Believe it or not, the point and purpose of testing is not to find defects with a product. The purpose of testing is to prove in a documented and repeatable way that a product does what it claims to do and exceeds customer expectation. Defect identification is a byproduct of proper test and design validation.
Why is this distinction important? Because it alters the perception of what we are doing on a day-to-day basis. It shifts the focus away from an unquantifiable and unachievable goal and puts the focus squarely where it belongs, on Product Validation.
On the one hand, if we are simply testing to look for defects then there is no purpose to having any organized systematic approach. We do not need to qualify or quantify our work; we simply launch the software or pick up the product and start pushing buttons and look for instances where something is broken. On the other hand, how do we know if it is a problem if we have not defined what correct behavior is?
When our focus is to prove that a product works as it is claimed to do, our job becomes much simpler. We now have goal and a target that we can focus in on and achieve. We can quantify and qualify everything we do with the end goal always in sight; proving that this product will do what we claim it will!
With this understanding in place we now have two very simple questions to ask. One: What is the product supposed to do? And two: How am I going to prove it does it?
To answer the first question of “What is the product supposed to do?” we turn to the sources of truth for the product, the Project Proposal, the Features and Specification Document or any other materials that define the product. First we discover all that is important and relevant to our product. From this we build our foundation, hence the need for these documents to be complete and accurate. When faulty data is supplied up front it should not be a surprise that faulty data is returned at the end.
Today’s modern QA practice allow us to ensure that we know exactly where we are going, how we will get there, and when we have arrived
The answer to the second question, “How am I going to prove it?” is a bit more challenging. There are myriad tools available to help with the collecting of data and creation of metrics. But at the end of the day the only metric that really counts is the perception that the end customer has. Is she happy with the product? If the answer is yes, then you have succeeded. If the answer is no, then you have failed.
Today’s modern QA practice allow us to ensure that we know exactly where we are going, how we will get there, and when we have arrived. This, then, is where experience, skill, and understanding come into play. This is where I, as a manager, look at the individual pieces and evaluate each for what it is worth. And then look at the whole and try and delve into the missing pieces and secrets that are hiding. Then, much as a stained-glass window artist will take seemingly broken and random pieces of glass and craft them into a cohesive whole, the results are a beautiful work of art that everyone can be proud of.