April 4, 2012

Stuff Breaks – And We Need to Test for It

Helix ALM
Test Management
As quality professionals, we’re all well aware that the applications we test are less than perfect. That’s one very important reason that the software quality field exists. Functional testing won’t make an application perfect, but it can help ensure a certain level of quality, and enable defects to be found and addressed prior to release. But it’s not just applications that break. The infrastructure surrounding the applications also breaks. Systems crash, networks fail, applications run low on memory, and drives get configured incorrectly. While failures such as these are outside of the scope of the application, they affect applications we test, sometimes severely. You might say that’s beyond the purview of application quality. But it’s not. The application is the primary way users are able to get utility out of computers and the infrastructure. If an application fails badly, users don’t care if the failure is the fault of the application or of the infrastructure behind it. They just know it let them down. That’s why we introduced stress testing as a part of Seapine ALM 2012. Stress testing in QA Wizard Pro allows you to simulate failures behind your application, both to see how the application responds  to mitigate potentially bad outcomes. With this release, QA Wizard Pro includes the ability to simulate low memory and storage conditions, a read-only drive, and failed networking while functional testing. These reflect real-life situations where the supporting infrastructure for the application may have failures or limitations. To learn more, watch this short video about how to use QA Wizard Pro’s stress testing features. You can add these stresses easily. First, click the Stress Recording toolbar button when recording a script. This opens up the dialog box for selecting your options. You can limit available memory, limit space on a drive or make it read-only, or disable networking for the duration of the test. After you select your options, click OK to close the dialog box, then record your script as you normally would. The stress options you set are applied when you run the script. QA Wizard Pro Stress Testing Dialog Box You can also add stresses to existing scripts as you record additional steps. You can manually add stress functions at specific points during script play back by adding stress statements to an existing script. The easiest way to do this is to record a new script with the stresses included, then copy and paste the stress statements into your existing script. The unique thing about stress tests is that you can’t otherwise simulate these kinds of failures or conditions using conventional testing tools. If you can’t do this kind of “what if” testing with your application, then your testing is incomplete. That’s not to say that your application isn’t of high quality, or not fit for its intended purpose. But if you want to make sure that your users have the best possible experience, you also have to ask questions about how your application works within its larger infrastructure.