October 8, 2015

Has All My Code Been Reviewed?

Surround SCM
Version Control
I’ve often had Surround SCM users ask, “How do I know if all code on my project has gone through a code review?” Maybe your process states that all changes must be code reviewed before releasing and you want to determine if it is safe to release. I’ve also had users say, “I think we are doing a good job with our code review process, but how can I know for sure?” Using many SCM tools you can find the answer to these questions, but it is a manual process that takes a long time. Starting with Surround SCM 2015.1, you can easily make this determination using the Code Review Coverage report.

Problem Scenario

Let’s say that a file is currently at version 1 and has been code reviewed. Joe checks in version 2 to implement Feature A, but he forgets to initiate a code review. Frank checks in version 3 to implement Feature B and adds the change to a code review container, but the code review is never completed. Sue checks in version 4 to implement Feature C and her code review successfully completed. How can you tell if all versions have been code reviewed? CodeRevCoverageHistoryScreen    

Looking at the Last Version is Not Sufficient

Some customers look to see if the last version of a file has been reviewed, but this approach does not always work. The problem is that most code reviews are only performed on the file changes, not on the entire file. If a five line change is made to an existing 1000 line file, performing a code review on the entire 1000 lines is inefficient if the previous contents already passed a code review. In the above scenario, the last version (version 4) has been code reviewed but the entire file’s contents have not been code reviewed.

Using Workflow has Limitations

Some customers use the Surround SCM workflow to track code reviews. This approach can work, but it has some limitations depending on how it is implemented. Each file has many versions but only has one current workflow state. In the example above, the file can incorrectly be marked as being in a Reviewed state when not all file versions have been code reviewed. Additionally a single workflow state cannot adequately convey complex situations such as the one above.

Using the Code Review Coverage Report

The Code Review Coverage report tells you which versions have been code reviewed and, more importantly, which versions have not. It can also tell you the name of the code review container that each file version is in, which is helpful when submitting information for an audit or when determining why something was missed. Following is the Code Review Coverage report for the problem scenario. Additional report columns can optionally be added to show more information. CodeRevCoverageReportScreen Version 2 is not included in any code review, so it is obviously unreviewed. Version 3 is in a code review but that code review was not yet completed, so it is also considered unreviewed. You can see version 3 is listed as an Unreviewed Version, but it is also displayed in the Code Reviews column in a code review that is still awaiting review. The Code Review Coverage report provides a comprehensive answer to the difficult question of “Has all my code been reviewed?” If you have been relying on reviewing the last file version or on workflow states to ensure that all code has been reviewed, you might consider changing your process to incorporate this report.

Report Options

Various filter restrictions can be applied to control the output of the Code Review Coverage report. Here are use cases for applying certain restrictions.
  • Restrict by date – Maybe you have not performed code reviews in the past, but you want all future changes to be code reviewed. Restricting by a date range can filter out all old versions.
  • Restrict by actions – If changes made on a child branch have been code reviewed on that branch, you could filter out Promote actions on the parent branch.
  • Restrict by filename – Your process may state that only C++ files need to be reviewed, so you can focus on just the CPP files.
  • Restrict by custom field – Maybe only a smaller set of source files are critical enough to require code reviews on all versions. You can create a custom field to identify those critical files and then restrict the report based on that custom field.
  • Restrict by user – You might be interested in making sure that all changes made by a new employee have been code reviewed.