My current engineering position focuses on testing. Proper testing requires both positive and negative test cases; that is, we must verify both that some things are present and that other things are not. How things work and how they don't work.
It sounds simple. Conceptually it is. In practice this attempts to bridge a major mental gap that afflicts a lot of people, myself included.
Software development is an ongoing process. Hardware changes, needs evolve, and the software that runs it all must keep up. When the software is modified it is my job to ensure that the change meets the specification without causing anything else to go wrong.
Verifying that changes have been made is the straightforward part. There is a list of the changes that have been made. If not a list, the programmer has a good idea. If the programmer has been laid off the changes between revisions can be examined. Verifying that the expected changes have been made at the expected points is not necessarily easy but it is straightforward.
Verifying that changes have not been made where they are not expected is trickier.
Say for example that you wanted to program an app that remembers your wife's birthday. It is a simple form with buttons with the names of all your family members. The positive test would be to click on the button with your wife's name. If you do so and it pops up the correct date, that is a positive test well done.
That doesn't mean it works correctly. Perhaps your programmer was both malicious & lazy and simply set the program to return your wife's birth date in response to all date queries. That would work just fine for the one button but all the other buttons would return your wife's birth date as well. Assuming that not all your family members were born on that same day this is a problem.
Proper testing requires verifying that the other buttons do not return the wrong date.
This sort of thing can fly out of control very quickly. Perhaps some software was written that worked perfectly under the original design scope but has been expanded to include more function. Implementation decisions that made sense in the original scope may be invalid in the new scope and cause code that hadn't been touched for years to be suddenly buggy.
It only gets worse from there.
This can be especially bad when the errors occur only on fringe cases so the programmers, who are focused on implementing the new code, aren't even made aware of their existence.
It is easy to look for what we expect and common to find it. This does not mean that the mental model of the situation is correct. It is important to develop the habit and discipline to attempt to falsify that model and verify that what is expected probably mirrors reality.
Probably.
Comments
Post a Comment