October 28, 2013

Taming Your Spec Depot to Keep It Under Control

Traceability

Image: by missresincup w/Flickr

The Perforce server's Spec Depot feature is very handy. If you don't already have a Spec Depot, you should!

It's simple to create one: simply use the 'p4 depot' command to create a new depot, and change "Type: local" to "Type: spec" to specify that your new depot is a Spec Depot.

From this point on, the Perforce server will automatically retain archived copies of user-edited forms. There's a great section in the System Administrator's Guide that describes this in detail: Enabling versioned specifications with the spec depot.

The nice thing about creating a Spec Depot is that it's very easy to do, and the information that the server retains for you can be very helpful.

However, if you aren't aware of it, the contents of your Spec Depot can grow, and you may find that you are retaining more information than you need.

There are a number of lesser-known features of the server which can help with this problem, specifically:

  1. Set "spec.hashbuckets".

    By default, the Perforce Server creates one subdirectory in the spec depot for each type of versioned spec, and beneath these, one subdirectory for each individual versioned spec. If you have a lot of specs of the same type, this can cause the spec depot to contain many files in the same directory on the server's filesystem, which has been known to cause performance problems.

    To avoid these limitations, use spec.hashbuckets to increase the number of directories available for specs. This configurable defines the number of buckets (subdirectories) into which files in the spec depot are hashed. By default, its value is set to 0; do not hash. To create 100 subdirectories into which spec depot files are hashed (and thereby increase the number of available sub-subdirectories by a factor of 100), set spec.hashbuckets as follows:

    p4 configure set spec.hashbuckets=100

    As of release 2013.3, the default for spec.hashbuckets is changing from 0 to 99, so soon this will happen automatically; but for now, if you don't already have spec.hashbuckets set, you should probably set it to 100.

  2. Set the "SpecMap:" field for your Spec Depot.

    By default, all specs are versioned. You can use the SpecMap:field to control which specs are versioned by adding lines in depot syntax that include (or exclude) paths in the spec depot.

    For example, you can exclude the protections table from versioning by configuring your spec depot's SpecMap:field as follows:

    SpecMap:
        //spec/...
        -//spec/protect
    
    In an environment such as a build farm, in which large numbers of temporary client workspaces and/or labels are created, you can configure the spec depot to exclude them, while keeping track of other changes to client workspaces and labels. For example, a spec depot configured with the following spec mapping:
    SpecMap:
        //spec/...
        -//spec/client/build_ws_*
        -//spec/label/temp_label_*
    

    will no longer track changes to client workspaces whose names begin with "build_ws_", nor will it track changes to labels whose names begin with "temp_label_".

    Note that adding or changing the SpecMap: field only affects future updates to the spec depot; files already stored in the spec depot are unaffected.

  3. Limit the number of versions of each spec that you retain.

    Providing a typemap entry that maps some or all of a spec depot's revisions to filetypes 'text+CS' or 'text+FS' will now cause the server to purge edit revisions of those spec depot files appropriately.

    For example, you could specify a TypeMap such as:

    TypeMap:
        text+FS8 //spec/client/...
    
    This will instruct the server to save only the most recent 8 revisions for Spec Depot entries for client specs.

Of course, the SpecMap and TypeMap specifications that you provide can be much more sophisticated than the simple examples I've provided here.

Although the "spec.hashbuckets" configurable has been around for many years, the "SpecMap" and "TypeMap" features that I described above are new to release 2012.1, as described in the release notes here:

        #360990 (Bugs #33080, #25377) **
            Providing a typemap entry that maps some or all of a spec depot's
            revisions to filetypes 'text+CS' or 'text+FS' will now cause
            the server to purge edit revisions of those spec depot files
            appropriately.

        #359100 (Bugs #15959, #24457, #31911, #36804) **
            If the optional SpecMap field of the depot spec for a spec depot
            contains a mapping, only revisions that match that mapping will
            be stored in the spec depot.

If you have a more complex scenario for managing your Spec Depot content, and want to know what techniques would work well, don't hesitate to contact Perforce Technical Support for more detailed guidance.

Have fun, and keep that Spec Depot tamed!