August 26, 2013

Fun with Formatting


Sharp-eyed users may have noticed the addition of a new undoc option to the command line client in 2013.2:

    p4 -F formatstring
        Override the formatting string of server messages, allowing output
        to be customized for easier scripting.  Use the '-e' global option
        to see examples of built-in message strings and the dictionary of
        variables associated with each message.  The '-F' option also works
        with the dictionary provided by tagged (-Ztag) output mode.

Although specifying a custom message format isn't something the average end user will want to do, it can make simple shell scripting much easier.

To use this feature effectively, it helps to be somewhat familiar with how Perforce's server messages work. Each message consists of a dictionary of name/value pairs and a string with variable names that are replaced with values from the dictionary. The "-e" global option (another undoc flag) can be used to examine a message to see how it's assembled.

 % p4 -e sizes Jamfile
 info: //depot/Jamfile#1 1502 bytes
 code0 335550918 (sub 454 sys 6 gen 0 args 4 sev 1 uniq 6598)
 ... code0 335550918
 ... fmt0 %depotFile%%depotRev% %fileSize% bytes [%blockCount% blocks]
 ... depotFile //depot/Jamfile
 ... depotRev #1
 ... fileSize 1502
 ... blockCount

 exit: 0

The first line (preceded by "info:") shows the formatted string that you would normally receive as output from this command. The "code" is the numeric identifier for this particular message (not very interesting for our purposes here), and the "fmt" is the formatting string. The remainder of the output consists of the name/value pairs that are substituted into the formatting string to construct the formatted message.

Turning the formatting string into a formatted message is very simple -- "%varName%" becomes "varValue". This message also contains a block in square brackets -- this block will be entirely omitted if the variable inside of it does not have a non-empty value. In this case the variable name is "blockCount", and since its value is empty neither it nor the word "blocks" after it is part of the formatted message. These bracketed sections can also provide alternate text to use if the variable is empty, e.g.:

 [%argc% - no|No] such file(s).

is a message that has slightly different text depending on whether a filename is given.

Not long after implementing this feature, I was tracking down a problem and wanted to quickly figure out what the largest file in a particular directory was. I could have eventually arranged the "p4 sizes" output into a readily sortable format with a bit of sed/awk/cut magic (remembering to be careful of spaces in file names, of course), or I could have written a short script using one of our scripting APIs, but the new "-F" flag makes for a very clean shell one-liner:

 % p4 -F "%fileSize% %depotFile%" sizes ... | sort -n | tail -n1
 27869735 //depot/the/path/bin/p4d

As you might already be envisioning, this also makes it trivial to pull file names out of one p4 command's output so that they can be redirected as arguments to another p4 command.

If you're familiar with the tagged output format (the format that "p4 fstat" uses, also enabled by the "-Ztag" global option on most other commands), you can combine that with the "-F" flag so that the variables are pulled from the same tagged dictionary that you're familiar with. Tagged output will sometimes provide additional values, or may have the same values in a more raw/unformatted form. For example:

 % p4 -F %change% changes -m1
 Change 646955
 % p4 -Ztag -F %change% changes -m1

Finally, one of the best things about this new feature is that it's purely client-side, so you even don't need to wait for a server upgrade to start using it. Download the 2013.2 beta now and give it a whirl.