March 30, 2010

Avoid Creating Stale Test Cases

Helix ALM
Test Management
Do you commonly write test cases for project requirements only to find out that you were looking at the requirement from last week and now the edit boxes are drop-downs and the radio buttons were turned into check boxes? So you have to go back and re-work the test but of course time isn't in the schedule so you're spending your evenings updating stale test cases. This huge headache is caused by a simple lack of communication. If someone had told you the requirements changed, you could have easily tracked down the latest copy and avoided the re-work. Hopefully you're using TestTrack TCM and TestTrack RM to manage requirements and testing on your project. If not, stop reading this post and start reading this white paper. For those of you using the TestTrack suite, there are two ways to manage changing requirements and avoid creating stale test cases. One way, what I call the "nuclear" approach, is to set up notification rules to update the team every time any requirement changes. This works fairly well in small shops where people are wearing multiple hats, but doesn't scale well. People stop paying attention when they get a change notice every half-hour and 90% of them are unrelated to their responsibilities.

Suspectivity

A more focused approach is to use Suspectivity flagging in TestTrack to automatically flag changes across all linked artifacts. In your requirements flow, add an event to mark items as suspect or "possibly changed enough that they bear review".

Note the last check box in the previous screenshot. That allows updating of all linked artifacts when a requirement is marked as suspect.

In Action

To put it in action, all you have to do is right-click a requirement and apply the event. The mark will propagate down to linked test cases and runs automatically. When you open the requirement to begin writing your test or a tester starts a test run, you'll be notified of changes and can begin your investigation.

Three quick best practice ideas:

  1. Add the Is Suspect column to your TestTrack list windows, it'll quickly show you flagged issues.
  2. If you're a test manager, create a tabbed view that shows you currently flagged items.
  3. The traceability matrix can quickly show you flagged items and their relationships, and it's filterable.

Make it Automatic

Once you understand the functionality, you can leave it a manual process or add some automation to the mix. Use an automation rule to mark a requirement as suspect any time it's changed. I wouldn't recommend doing that for every requirement, so use a filter on the automation rule to only catch certain requirements. There are several possible filters you could use.

  • Only requirements with existing links to test cases or test runs.
  • Only certain types (e.g., functional requirements)
  • Use a custom check box for minor changes and let the individual make that decision.