March 14, 2013

Long Live Black Box Testing!

Test Management
[caption id="" align="alignright" width="74"]Bart Horning Bart Honing, Quality House[/caption] This is a guest post from Bart Honing of Quality House. Bart has been working for over 7 years in the field of testing, from developing and implementing test cases to test management in various industries ranging from banking to government.   Since I started off as a software developer, I have a tendency to dive into figuring out how the black box actually works on the inside. This attitude can be of good use when working on test analyses and test execution. But, when starting off on a project aimed at integrating black box components, it can be a challenge to focus on the interfaces instead of the logic within the box. With the increasing complexity of software and the desire to integrate services, black box testing—or system testing—is far from obsolete. Black box testing can be applied to many types of test objects and in various stages of testing—no surprise there. Applying black box techniques is, however, a challenge for many testers. The main challenge lies in creating a testing environment that not only verifies functionality but actually works as the surrounding process in which the object under test is going to function. Creating a representative testing environment that takes into account all the influences that can interact with the black box can, for instance, be supported by simulators or stubs. Alongside the technical influences the tester can try to take into account the implicit outcome expected by the business. This sounds very high-level but I believe as a tester you can, and perhaps need to, reflect on the bigger picture in which you execute your test cases. I like to know the actual usefulness of a project or the process in which a tool needs to be implemented to determine whether what I am testing actually is needed or doing what it is supposed to do. As a testing technique, black box testing is often used to determine whether the object meets its expected criteria. When managing a black box test, results alone don't say much about the technical quality. When discussing the implementation of a new piece of software with a business manager, he pointed out that it didn’t matter how the outcome was created as long as it suited the business needs. Because he was putting his money where his work was, we were limited to determining if the black box was the right tool for the business process. We gave the supplier a bit of information and determined by the feedback and analyses of the black box outcome if we were getting value for our money. To me, this was the fine line a tester needs to walk to not only agree to the limited business demands but to also be able to actually say something about the quality of the black box outcome. By being critical and being able to convert your technical knowledge into quality feedback you can use black box testing as a valuable instrument. One other aspect that can apply and which is not always taken into account is the suitableness of the black box. A determination of whether the black box can be used in combination with manual or existing operations can actually increase the quality level of the test result. Getting this information, when applicable, or any other sort of information on the context in which the object under test will be used gives the tester the opportunity to step outside his or her own comfort zone and learn. This also is a movement much more in support of modern agile development teams where the role of the tester is more cross-domain focused. The challenge with black box testing is the ability of the tester to not only work with the possible limited opacity towards the test object but look beyond and determine if the object fits in with the process that it is meant to support. Is black box then still suited as a testing technique? Yes, but it needs to be approached in a manner in which you can really say something about the role the object (tool, class, component) fulfills under various circumstances.