May 17, 2011

Streams: Taking the "You" Out of Views

Branching
Streams

As promised, here's your introduction to stream views. This is a feature that's easier to use than it is to explain, so I'm going to demonstrate with examples. For those of you who are new to my series, the disclaimer: Perforce streams won't be available until later this year.

First, a little background. Perforce is designed to version large file sets. The typical Perforce repository stores far more files than you would need on disk for the work you do. Perforce "views" filter the repository files so you can see and work on only the files you need. There's your "client view", which maps your workspace to parts of the repo. And there are what we call "branch views", which map one set of repo files to another for diffing and integrating. ("Integrating" being Perforce's categoric word for branching, copying, and merging.) If you know how views work, you can use them to effect all kinds of fantastic environments and transformations.

Streams take the "you" out of views. The Perforce Server generates workspace views and branch views for you, from stream specs. As a user, you may never see these generated views. In fact, if you're not already a Perforce view expert, you many be wondering why I'm taking the time to blog about stream views. The reason is that a lot of Perforce experts are dying to know more about them. So, for you experts out there, this post is for you.

A stream spec contains a template from which views are generated. The template is much simpler than the views themselves. Even better, the generated views are inherited by child streams. Creating a stream doesn't mean you have to create a template -- your stream simply inherits its parent's view. And if the parent stream's view doesn't suit your needs exactly, you can edit your stream's spec and customize its view template.

Let's take a look, starting with a very simple case. Here we have two streams, //Ace/main and //Ace/dev, along with their specs:

What you see in both specs is the default view template -- the entire stream path is "shared". (I'll explain this in a bit.) How does this affect client views? Well, when you switch your workspace to the //Ace/main stream, its client view becomes this:

The client view maps the root of //Ace/main to the root of your workspace. Same thing happens after switching your workspace to //Ace/dev -- now its view is this:

As I mentioned in a previous post, you can use commands like p4 copy -S //Ace/dev to integrate between a stream and its parent. The -S flag says "use this stream's branch view", the view that maps the stream to its parent. //Ace/dev's branch view looks like this:

This is due to the default view template, which says the entire stream path ("...") is to be shared with client views and branch views. Because of that, the entire stream can be synced to workspaces, and the entire stream can be branched and integrated.

This is simple, easy to use, and perfectly suitable for small projects. For large projects, it isn't practical, of course. So, what are the alternatives?

Streams offer a choice of behaviors for each path (and by "path" I mean any level of subdirectory):

  • Share changes with other streams. These paths appear in both client views and branch views, and are mapped to the stream. The effect, as I just mentioned, is that files can be synced, submitted, and integrated.
  • Isolate changes from other streams. These paths appear only in client views, and are mapped to the stream. That means you can sync and submit the files in these paths, but you can't integrate them. This is useful for built binaries, generated docs, and other content that you want checked in but not branched or merged.
  • Import a path from elsewhere. These paths appear only in client views, and are mapped per the parent stream's view (the default), or to another depot location. You can sync these files, but because they're not mapped to the stream itself, you can't submit them. Nor can you integrate them. All they do is mirror other paths.
  • Exclude a path from both client views and branch views. This keeps unneeded content out of workspaces and streams entirely.

Here's an example from Massively Large Media Corporation, a fictional company. MLMC is a conglomerate of three separate software companies: Acme Systems, Red Resources, and Tango Tools. Each company makes a sizable product, and all three contribute to MLMC's bundled products. The files for these components are housed in three separate depots, //Acme, //Red, and //Tango.

The Acme mainline is configured thus:

I've color-coded the view template so you can recognize the effect each path has on the generated views. Here, for example, is the client view you'd get if you were to switch your workspace to the //Acme/main stream:

It's pretty easy to see what's going on here. The stream spec's Paths field lists folders relative to the root of stream. Those are the folders you'll get in your workspace, beneath your workspace root. The shared folders are mapped to the //Acme/Main path, and the imported path is mapped to its location in the //Red depot.

As it turns out, no one does actual development in the Acme mainline. Take the XProd feature team. They have a development stream of their own, defined thus:

Switching your workspace to the //Acme/XProd stream gives you this view:

Here we see stream view inheritance at work. Imported paths are mapped to whatever they're mapped to in the parent's client view, unless they are given other paths to import, as in the case of tools/tango. The shared and isolated paths are mapped to the child stream; these contain the files the XProd team are working on and will be submitting changes to. And the excluded path doesn't appear in the workspace view at all.

Because the //Acme/XProd stream has a parent, it has a branch view that can be used by the copy and merge commands. That branch view is:

Wow! All those paths in the //Acme/XProd view, and only apps/xp can be branched? That's right -- it's the only path that is shared in both streams. When you work in an //Acme/XProd workspace, it feels as if you're working in a full branch of //Acme/Main, but the actual branch is quite petite.

Now, let's go one level deeper. Suppose Bob, for example, creates a child stream from //Acme/XProd. His stream spec looks like this:

Note that Bob's stream has the default view template. Given that Bob's entire stream path is set to "share", will his entire workspace mapped to his stream? No, because inherited behaviors always takes precedence; sharing applies only to paths that are shared in the parent as well. A workspace for Bob's stream, with its default view template, will have this client view:

As you can see, a workspace in Bob's stream is the same as a workspace in the XProd stream, with one exception: the submitable paths are rooted in //Acme/BobDev. This makes sense; if you work in Bob's stream, you expect to submit changes to his stream.

By contrast, the branch view that maps the //XProd/BobDev stream to its parent maps only the path that is designated as shared in both streams:

The default template allows Bob to branch his own versions of the paths his team is working on, and have a workspace with the identical view of non-branchable files that he'd have in the parent stream. And Bob didn't have to lift a finger to create it!

I hope these simple examples were clear, yet varied enough to hint at the kinds things you can do with stream views. One key point is that child streams can't branch more than their parent streams are willing to share. Another is that stream views default to the parent view. This gives control to owners of parent streams without completely tying the hands of developers who branch their own child streams.

The Paths field is only one part of the stream view template, by the way. The template also offers a way to define remapped and ignored files, the effects of which are inherited by child streams. You'll be happy to know, for example, that a mainline can be configured to forbid anyone, in any related stream, from checking in files with names like ".*.swp"!

As always, I welcome your comments and questions. I do read them all, and I plan to roll them up into material for subsequent posts.