November 4, 2013

What changes haven’t been reviewed and approved? Part II – The Groovy implementation

Community
Helix Swarm

Image: Zeusandhera via Flickr

What are we using this time?

In the first part of this article I explained why to have a report of all changes not related to an approved review for a given branch in Perforce. Now it is time to implement such a report and we need to get our tooling sorted. For the task at hand we need to query Perforce and Swarm data, and although you could use almost anything, I’m picking Groovy as the scripting language. Wait, Groovy? You’re thinking there isn’t a P4Groovy, is there? Well, we don’t need one. Groovy is relatively new, based on Java and can therefore make calls to P4Java easily. This is particularly useful, as we don’t need to download and compile any platform specific library as with P4Python or any other Perforce API.

Groovy also has access to libraries that make parsing JSON data a snap and it has nice built-in support for set operations.

Getting the sweet grapes

We need 2 libraries that do not ship with Groovy out of the box. The first one is P4Java of course and the second one is the convenient RESTClient class that comes with the http-builder package provided by codehaus.org . These libraries need to be downloaded and added to our Java execution classpath. This is an area where Groovy starts to shine and with so-called Grapes (The Groovy Adaptable Packaging Engine) it’s a no-brainer.

import com.perforce.p4java.server.*
import com.perforce.p4java.impl.generic.core.file.*
import groovyx.net.http.RESTClient
@Grab(group='com.perforce', module='p4java', version='2013.1.656196' )
@Grab(group='org.codehaus.groovy.modules.http-builder', module='http-builder', version='0.6')

The @Grab annotation will ensure the libraries get download from a public Maven repository and they will conveniently be made available to the classpath for the script.

What set of changes is in Perforce?

Getting the changes from Perforce isn’t much harder either. We need to know P4PORT of course and need to connect as a user.

def server = ServerFactory.getServer( "p4java://p4demo13-1:1666", null)
server.setUserName("Joe_Coder")
server.connect()
server.login(null)

Once we are connected, getting access to the changes of a given branch in Groovy is 2 lines of code

def fileSpecs = [new FileSpec("//depot/Talkhouse/main-dev/...")]
def changes = server.getChangelists(0, fileSpecs, null, null, true, false, false, false)

Retrieving and parsing JSON formatted review data from Swarm

The RESTClient allows Groovy to talk to a Swarm instance. An HTTP GET to the review URL of a given Swarm project (here Talkhouse) with the request query being format=”json” will retrieve the review data and format it nicely in the response object.

def getRESTClient() {
        return new RESTClient("http://p4demo13-1/")
  }
def swarm = getRESTClient()
def response = swarm.get(path: "project/talkhouse/reviews" , query : [format:"json"] )

The RESTful API won’t just return to you an eventually massive dataset in one go as the Swarm website would just populate the visible area of the page. Therefore you will want to set the additional query parameter “max” and “after” in order to retrieve data in consumable chunks.

def getReviewIds(server, bulksize, last, swarmChangeIds, receivedReviews){

        def response = server.get(path:         "project/talkhouse/reviews" , query : [format:"json", max:bulksize, after:last] )
        response.data.reviews.each {
     
           ...
        if (response.data.lastId > -1) {
                last = response.data.lastId
                getReviewIds(server, bulksize, last,swarmChangeIds,receivedReviews)
        }
}

getReviewIds(swarm, 1, null, swarmChangeIds, 0)

A little set operation and you have it

Groovy has support for set operations and as a former database guy I’m always excited about it. This loops over the changes we retrieved from Perforce and creates a set of all changelist numbers.

def p4ChangeIds = [] as Set
changes.each {p4ChangeIds.add(it.getId())}

And this one loops over the reviews from Swarm and also creates a set of changelist numbers by parsing the JSON data and restricting the set to those parts of reviews that are in the Approved state. We need that inner loop as a review can have many changes. This code would go into the function to retrieve the reviews listed above.

response.data.reviews.each {
        if (it.state.contains("approved")){
                it.changes.each {
                        swarmChangeIds.add(it)
                        }
                }
        }

Finally we simply remove the second set from the first one and print the output as Swarm URLs.

Getting back to the first part of this little article series, we should strive for a rather short or empty list in our development process.

p4ChangeIds.removeAll(swarmChangeIds)
println "all p4 changes that are not covered by approved Swarm reviews"
p4ChangeIds.each {println "http://p4demo13-1/changes/" + it}

Read part one of this series: Finding the Balance of Freedom and Control in Code Review