Eight Attributes for Continuous-Test Systems
Within an Agile and DevOps environment, Continuous Testing (CT) is the proof of successful merges and continuous integration. At MERGE 2014, the Perforce Conference, I presented our experiences and the best practices we've put in place at Spirent Communications for Continuous-Test (CT) systems. From these experiences and best practices, we have defined the eight attributes necessary for Continuous Test Systems:
- Test selection and configurations are change-driven and automatically triggered according to changes committed to the integration trunk branch in Perforce.
- The entire test system is scalable from individual desktops all the way to global distributed labs.
- Both real physical lab equipment and scalable on-demand virtualized testing systems are supported.
- Dashboard support Key Performance Indicators provide visible build and test status for the entire development, test and release teams.
- Restful services are needed for test creation (Create), test set-up (Connect), test execution (Play) and test results analysis functions.
- Pre-packaged tests for standard components are available.
- Variable skill levels are supported so the entire team can create tests for all code changes.
- Automated test result analytics support continuous change controls.