June 3, 2009

P4Bucket

E-Commerce
Integration
What's New

AlienBrain(R) has a feature called bucketing that allows archiving of assets from the configuration management system to external storage. This saves resources and can increase performance - certainly as far as backups and restores are concerned.

Perforce does not have a feature like this (yet). There is the possibility to use the +S<n> filetype to keep only a limited number of revisions around, but the problem is that older revisions will be purged, that is, completely removed from the server, with no easy way to recover a purged revision.

In response to a customer request I wrote a script called P4Bucket that can be used to archive and restore binary depot files. The script is in the public depot.

There is also a presentation in PDF format.

The idea is to define one or several buckets (name and location, really), then use the buckets to archive files, which will be replaced by a small place holder (a "ghost"). Only binary files can be archived, and lazy copies or files that are sources of lazy copies will be ignored. Here is the command line syntax of the tool:

p4bucket command [options]:

init                        - initialize the connection parameters
create <name> -r <root>     - create a bucket <name> with root <root>
edit <name> -r <root>       - change the root of bucket <name> to <root>
delete <name>               - deletes the bucket (must be empty)
list [name] [-a]            - list bucket information
option -a lists contents

archive -b <name> [-n] [opts] file[revRange] ...   - archives the files
restore -b <name> [-n] file[revRange] ...          - restores the files

opts are
-f      forces archiving of head revisions
-s      snaps lazy copies before archiving
-m size minimum size for archiving files
-n      do not archive or restore, instead report the files

P4Bucket can be run in reporting mode to see how much space would be saved. Note also that the head revision can normally not be archived unless the "-f" option is used - for obvious safety reasons.

The tool is written in Python and requires P4Python 2008.2 or higher to run.

Have a look at my Perforce user conference 2009 presentation Adventures in Scripting Land, which shows some of the details of the implementation.

Happy hacking.

Sven Erik