November 5, 2014

Eight Attributes for Continuous-Test Systems

Continuous Delivery
MERGE User Conference

Automated Testing in Continuous Change Management

Within an Agile and DevOps environment, Continuous Testing (CT) is the proof of successful merges and continuous integration. At MERGE 2014, the Perforce Conference, I presented our experiences and the best practices we've put in place at Spirent Communications for Continuous-Test (CT) systems. From these experiences and best practices, we have defined the eight attributes necessary for Continuous Test Systems:

  1. Test selection and configurations are change-driven and automatically triggered according to changes committed to the integration trunk branch in Perforce.
  2. The entire test system is scalable from individual desktops all the way to global distributed labs.
  3. Both real physical lab equipment and scalable on-demand virtualized testing systems are supported.
  4. Dashboard support Key Performance Indicators provide visible build and test status for the entire development, test and release teams.
  5. Restful services are needed for test creation (Create), test set-up (Connect), test execution (Play) and test results analysis functions.
  6. Pre-packaged tests for standard components are available.
  7. Variable skill levels are supported so the entire team can create tests for all code changes.
  8. Automated test result analytics support continuous change controls.

Watch the 30-minute presentation to get more details. To learn even more, read my paper from the 2011 Perforce User Conference paper that describes my continuous change-driven test methodology.