May 16, 2011

Webinar Recording: Take the Pain out of Test Case Management

Test Management
Thanks to everyone who joined us for our Take the Pain out of Test Case Management webinar. The recording is now available on YouTube if you missed the webinar or want to watch it again.  It is also available on SlideShare if you want to view or download the slide deck. Q&A from the session follows.


Is there something that could help if the requirements or design are incomplete during a release cycle? I saw one suggestion to try to "Test earlier in the development cycle and reduce rework." But what if this depends on environments being updated too? Incomplete or changing requirements are one of the realities faced by any development team. Managing change is one of the motivations behind the growing popularity of Agile development methodologies. This question raises a good point. Incomplete requirements can be a more difficult burden on testers than developers because testing has to be mapped to the requirements. If requirements are vague or incomplete, it’s a challenge to write and execute tests. Testing earlier in the development lifecycle won’t help, because it simply increases the likelihood that those tests will need to be rewritten and re-run anyway. I can offer two ideas here, and I suggest using both of them. The first is to do more to stabilize the requirements and design before writing any test cases. That may mean more effort by the business analysts to better define needs, or it may mean that the testers go back to the users with questions to obtain more specific information. The second is to simply acknowledge that there will be changes to requirements after the project begins, as the ambiguities become apparent and the problem domain better understood. You have to be ready for these changes, and have tools and procedures in place so that changes and additions to requirements readily flow to all of the downstream artifacts. How do you know how large a test case you should write? Should you just write a test case for a requested change? Or also mention what other tests this change can affect? As requirements change, you have to look at each affected test case individually. TestTrack will automatically flag suspect test cases based on upstream changes to requirements, letting you zero in on the artifacts that may require changes or additions. How you make these changes depends on the artifact, the extent of the change, and when the change occurs. If you have confidence in the underlying functionality, and only the position of a button on the screen has changed, this may not require a full testing regime. You have to use judgment here--the answer isn’t the same for all situations. One thing you might do is regenerate test cases based on those changes, and compare them to the existing test cases. While it’s likely you will have to add some details to the regenerated cases, it will give you an idea of the extent of the change and what might have to change in the original test case. Do we need to include time spent for testing and test case development in looking at a requested change, or this is tracked separately? As requirements change, the amount of time you spend testing is also going to change. TestTrack keeps track of the testing effort and time required, so you want to make sure you factor in your estimate of time to make the changes to downstream artifacts, as well as the additional time needed to perform additional testing. Don’t shortchange yourself and the testing team by not raising the likelihood of needing additional time or resources in response to a change. Just as importantly, significant changes may impact the entire project schedule. You want to make sure this visibility exists in TestTrack so that both the project team and management can see the schedule impact of any proposed changes. How do you ensure that all scenarios/test runs of a test case were executed?­ There are several ways this can be done in TestTrack. If you are viewing the details of an individual test case, clicking its Test Runs tab will let you see the status of all of its test runs. The total number of open test runs is also an attribute of each test case that can be easily used to monitor the status of test runs. Additionally, several test case reports let you verify test run execution status. Is TestTrack RM, the requirements module of TestTrack, required for test case management? No, you can use the test case functionality in TestTrack TCM without purchasing TestTrack RM. However, having both gives you traceability and automation capabilities that otherwise aren’t available. For example, trace matrices and impact analysis reports are available when requirements and test cases are linked, which is done automatically if you have both products. Can you order test cases so the tester knows which test case to start with and the order of the following ones? Yes. An easy way to do this is to use folders. Items in a folder can be ordered or ranked. Will TestTrack run automated testing scripts via an automation tool? TestTrack TCM integrates seamlessly with QA Wizard Pro to support automated testing. If you use another automated testing or scripting tool, you can also schedule and run automated script files through the use of a separate executable, known as a script agent, and a text file for communication between the script agent and TestTrack TCM. Read more and download the script agent here. Does TestTrack TCM have support for 'higher' level test planning, such as test plan documents, testing prioritization, etc.? A common scenario is to use the flexible folder structure to represent the testing plan and organize testing artifacts. Can TestTrack be integrated with TSF? Yes, there is integration between TestTrack and TFS, but it’s not going to be as seamless as what you would see with Surround SCM and TestTrack integration. Does TestTrack support defect lifecycle management? Yes, TestTrack supports issue management, test case management and requirements management in one integrated platform. Does TestTrack TCM support importing existing test cases from a spreadsheet? Yes, you can import test cases from Excel. Is it possible to make relations between the workflow in TestTrack TCM and in TestTrack Pro?­ Not currently. Items in TestTrack TCM and TestTrack Pro maintain their own workflows. Can you create test cases without requirements? For instance, if someone asked you to do ad hoc testing, would you still need to create test cases? Yes, you can create test cases without associated requirements. Test cases and requirements are separate object types that can be optionally linked in TestTrack. You can create a test case from a requirement or create a test case on its own.