November 24, 2009

Automating folder replacement using P4Java and Apache Ant


Our internal build system for P4Java and P4WSAD does a lot more than just compile. It runs Checkstyle for static code analysis, JUnit for unit testing, EMMA for code coverage of those unit tests, generates various code metric reports, and last but not least generates Javadoc documentation.

Each of these tools generates one or more reports and what results at the end of each build is a folder of reports to submit to the depot. The problem is that these tools can consist of different sets of files from build to build with different names and paths.

So the process for getting them into the depot while cleaning up files left over from the last build is the following:

  1. Checkout files that are present in both builds.
  2. Add files that are new in the current build.
  3. Delete files that were present in the last build but no longer in the current build.

A good example of how this can occur is when generating Javadoc documentation. If you move a Java class between packages then the next time you generate the documentation the HTML file for that class will be in a different location in the Javadoc folder hierarchy. You want to delete the HTML file at the old path and add the HTML file at the new path so if, for example, you host your product's latest Javadoc using P4Web then you ensure you have the latest documentation only and not a sum of all the Javadoc files that have ever been generated and checked in.

This type of problem was easily solvable by writing a custom Anttask using P4Java that would analyze the local and depot folder and perform the proper adds, edits, and deletes and then submit the results as a single changelist.

The Perforce Replace Ant task is designed to automate folder replacement and can be configured with a Perforce user, client, and server address, and a changelist comment and then run from any Ant build file. The task is setup to first copy a folder from one directory to another so that you can keep your builds directories and your Perforce client workspace folders separate.

The following commands are run to analyze how the content of the local folder differs against the latest in the depot:

  1. p4 add for all files in the directory
  2. p4 diff -se to find files that need to be opened for edit
  3. p4 edit of the files returned from the previous p4 diff -se command
  4. p4 diff -sd to find files that need to be opened for delete
  5. p4 delete of the files returned from the previous p4 diff -sd command
  6. p4 revert -a to revert any unchanged files
  7. p4 submit of all the files opened for add, edit, or delete

The Perforce Replace task submits the changelist it creates at the end of opening files for add, edit, and delete and uses a default comment if none is specified. This task is based on P4Java and is a good example of how that API can be used for writing Java-specific Perforce extensions and applications. P4Java can be downloaded from the Perforce API tools site.

Below is an example of how the Perforce Replace Ant task can be used. It supports the P4PORT, P4CLIENT, and P4USER environment variables or those values can be specified either using properties or attributes to the task. It also requires a from and to directory to be specified to initially copy the files. The source can be found in the Perforce public depot as and the example file as replace.xml.

<projectname="Perforce Replace Task Demo"default="main"><taskdefname="p4replace"classname="com.perforce.ant.tasks.PerforceReplaceTask"classpath="p4replace.jar;p4java.jar"/><propertyname="p4.port.replace"value="localhost:1666"/><propertyname="p4.client.replace"value="build_client"/><propertyname="p4.user.replace"value="build"/><propertyname="reports_depot_dir"value="/home/build/client/builds/reports"/><propertyname="reports_build_dir"value="/tmp/build"/><propertyname="reports_comment"value="Updating build reports"/><targetname="main"><p4replacetoDir="${reports_depot_dir}"fromDir="${reports_build_dir}"comment="${reports_comment}"/>        </target> </project>