June 17, 2009

The 2009.1 Release


It's (beta) release-time again! Since the last release, 2009.1 brings us updates to all of theusualsuspects.

I make use of the nightly builds out of our main development branch for my day-to-day work, so I've had a chance to see the evolution of the new features and to get an idea of what sort of things you can do with them. When reading the relnotes, it can be a bit of work to distill what's new and where it is in the product, so to save you some time and relate a few of my own observations I skimmed through the list of changes and put together a few examples and added commentary to help tie things together.

- When displaying a two-way diff, P4Merge lets you edit the content of an opened-for-edit file, so now when you do that final diff-before-submit to check that everything is nice and tidy, you can correct errors in-place rather than going back to your editor, saving, and re-diffing.

P4Merge inline editing

- P4V's Administration tool lets you create/edit depots via the Depots tab. It's one more thing that you don't have to go to the command-line to do.

P4V Admin depot editing

- P4V can now have functionality turned off or configured via some simple XML files. To get an idea of the sort of things you can do, take a look at one of the sample files. Here's what's in my copy of "p4vdir/lib/p4v/P4VResources/examples/p4vfeatures.xml":

<?xml version="1.0" encoding="UTF-8"?>
 <FeaturesEnabled varName="FeaturesEnabled" >
 <Visibilility varName="Functionality" >

If you have users who don't care about various features, now they never have to see them. Hopefully it'll make it easier to get up to speed.


- Perhaps the flashiest two new features in P4V are the ability to preview audio/video files on the Mac and Windows platforms, and the ability to render HTML pages from the depot. Here's a sample HTML preview from one branch of our web site. Also, if you've got the whole thing synced, you can click on the links and navigate around!

P4V html preview

- Now to move on to P4/P4D!

- For added flexibility in accessing the server's log file, you can now fetch it remotely with the "p4 logtail" command. It accepts offsets in the log file, and tells you about where it stopped, allowing you to reliably get the data to the client, and to restart from where you stopped. There are lots of potential uses for it, especially if you don't have ready access to the server machine. Take a look at Track2sql or the online log analyzer for some idea of what's possible with this data.

Here's a quick example. First, grab the whole log file and save it as "log":

p4 logtail -s 0 > log

Now look at the first few lines of what you received. Notice the two pieces of tagged data that precede the actual content.

... file log.txt
... data
Perforce server info:
2008/05/08 11:30:22 pid 13927 [email protected]_____CLIENT_UNSET_____ [Perforce Visual Client/v63 (brokered)] 'user-info'
Perforce server info:
2008/05/08 11:30:23 pid 13927 completed .155s 0+2us 0+0io 0+0net 0k 2pf

There's also more metadata at the end of the file:

--- total lock wait+held read/write 0ms+103687ms/0ms+0ms
--- db.revsx
--- total lock wait+held read/write 0ms+103687ms/0ms+0ms

Perforce server info:
2009/06/11 13:09:59 pid 32162 [email protected] [p4/2009.2.main/LINUX26X86_64/198120] 'user-logtail -s 0'
... offset 206537370

- A feature mainly useful for people writing applications with our API (or for those who want to minimize the amount of data sent over the wire) is a new flag to "p4 fstat" that instructs it to only send the data fields that you're interested in. Previously, you got what fstat wanted you to get:

p4 fstat -Ol //depot/case/*
... depotFile //depot/case/File
... clientFile /2/work/u/5/depot/case/File
... isMapped
... headAction add
... headType text
... headTime 1158190775
... headRev 1
... headChange 912
... headModTime 1158190771
... fileSize 5678
... digest 841D8CE98F00B204E9800998ECF8427E

... depotFile //depot/case/file
... clientFile /2/work/u/5/depot/case/file
... isMapped
... headAction add
... headType text
... headTime 1158190775
... headRev 1
... headChange 912
... headModTime 1158190771
... fileSize 1234
... digest 041D8CD98F01B204E9800998ECF8427E

Now it listens to your concerns, and answers appropriately:

p4 fstat -Ol -T 'depotFile,fileSize' //depot/case/*
... depotFile //depot/case/File
... fileSize 5678

... depotFile //depot/case/file
... fileSize 1234

- Perhaps the big feature in 9.1 was something that caused the first substantive update to the "p4 help rename" text since it was first checked in. "p4 move" is really handy, and will probably cause you to blurt out "I like to P4-move-it, move-it. I like to P4-move-it, move-it!" Basically, it takes the integ/delete/submit combination and moves it into a single command which can be repeated multiple times before submit.

Here's a quick example of renaming a directory tree:


total 4.0K
drwxr-xr-x 6 jgibson jgibson 64 2009-05-11 19:23 ./
drwxr-xr-x 10 jgibson jgibson 4.0K 2009-06-11 13:26 ../
drwxr-xr-x 3 jgibson jgibson 20 2009-03-11 13:46 include/

Edit, move:

p4 edit include/... > /dev/null
p4 move include/... exclude/... > /dev/null


total 4.0K
drwxr-xr-x 6 jgibson jgibson 64 2009-05-11 19:23 ./
drwxr-xr-x 10 jgibson jgibson 4.0K 2009-06-11 13:26 ../
drwxr-xr-x 3 jgibson jgibson 20 2009-03-11 13:46 exclude/

Now that may not seem so impressive, but after having used it for a few months, I can tell you that it's quite handy. It lets you do a lot of client-side shuffling before a submit. When our main server was updated with move, I immediately did a bunch of renaming that I had been putting off.


- Now, the last feature I'll show you is a new storage filetype; +X, not to be confused with the existing lower-case +x. Capital +X, or the "archive trigger" is a type that lets you write your own code to manage file content, server-side. Whenever a file is marked as +X, the server delegates all responsibility for that file's content to the associated trigger program.

There are some interesting things you can do with this that I'll illustrate in a future post, but the intended use is to make it easier to offload file content from your server machine. To use this, there are three parts that must be in place:

  1. A file marked as +X
  2. An associated archive trigger definition
  3. The archive trigger code.

So, the way this works is that when a client requests some content via sync/print/etc (or sends it with a submit), P4D looks to see if the file is +X and if there's a matching archive trigger for it. When it finds a match, it runs the trigger, which gives the server the content which is then piped on back to the client. The end-user sees no difference.

To get started you should know how it all goes together and what information the trigger has available to it. Archive triggers get their metadata as command line arguments, read file content on STDIN, and write it via STDOUT. An easy way to test it out is to write a trigger that just echos back what it received, like in this Perl snippet (3):

while( <STDIN> ) { chomp; } # eat any content P4D sends us for 'read' ops.
print "@ARGV\n"; # send back our parameters as file content to the client.

Then when you've got that working, add a trigger definition like the following (2):

archive_testing archive //depot/file "/p4/triggers/archive_tester.pl %op% %rev% %file%"

See "p4 help triggers" for more details. Quoting from it, the arguments are:

%op% -- read/write/delete for archive access
%file% -- name of archive file
%rev% -- revision of archive file

Now add a new file (1):

echo "12345" > file
p4 add file
p4 submit -d "The file's content is '12345'." file
p4 edit -t +X file
p4 submit -d "The file's content is now whatever the archive trigger gives us." file

So now when you run "p4 print -q //depot/file#2", p4 will print out "read 1.2 depot/file" as the content of "file", since that's what the trigger printed out. Run "p4 print -q //depot/file#1" to see "12345".

It's worth noting that the file path is not a normal depot-syntax path. Rather, it's related to the depot definition's "Map:" field. Run "p4 depot -o depot" to see yours.

The following two examples show what differing Map fields do to the file variable.

Depot map relative to the P4ROOT:



p4 print -q //depot/file
read 1.1 depot/file

And an absolute path:



p4 print -q //depot/file
read 1.1 /disk2/depot/file

And that's really all there is to it. Your trigger logic can do anything it wants, from fetching a file remotely via the network, mailing a clerk the location of a hard drive in the middle of a warehouse, to moving a BIG ROBOT ARM:

Fermilab tape robot; http://www.fnal.gov/