March 17, 2011

Streams: The Big Picture


Until now, Perforce users have relied on externally defined road maps for wrangling codelines. Soon we will have the option of using native Perforce "streams" for that. In today’s blog I’d like to paint the broad, high-level picture of streams. I’m not going to go into too much detail -- I’ll focus on that later. For now I’ll just touch on key points.

The raison d'être of streams is to model branching and merging intent. Streams do that through their parent-child relationships. So one way to look at streams is with a flow diagram:

What you see here is a mainline and the streams in its lineage. Changes are intended to flow along the connectors between streams. The intended flow of change can be changed by changing the stream lineage. I say “intended flow” because it still takes a user action to branch and merge files. However, the Perforce commands for branching and merging will soon be stream-aware, with defaults that support intended flow.

In the Perforce model, streams have what we call "stability". Development streams are less stable than their parents; their role is to keep the churn of active development from destabilizing code that is or will soon be released. Release streams are more stable than their parents; they assure that the exacting tasks of the release pipeline don't hold up active development. In the middle is the mainline, which provides ultimate control over all the streams related to it.

In the diagram, the more stable stream is always shown above its less stable parent or children. This tells two stories. First, it tells us that the higher up a stream appears in the diagram, the riskier it is to do work in it. Second, it tells us how to propagate change along the stream connectors: merge down, copy up.

So that's the model. (Does it sound familiar? You heard it in The Flow of Change.) Now, let's talk implementation.

Stream specifications are stored by the Perforce Server, just as specs for workspaces, users, groups, depots, and other Perforce objects.  A stream spec controls files in a codeline -- in fact, a stream’s unique ID is the  depot location of the files it controls. Stream identifiers are immutable, and because of that, streams also have owner-modifiable names. Here's an example of a stream spec:

Stream:	//Ace/DEV
Parent:	//Ace/MAIN
Owner:	jack
Name:	Diamond Dev
Type:	development
	Shared development stream for Diamond features.

The stream spec also controls two internal views. One view defines the scope of workspaces (we’ve been calling this the “logical view” of the stream), and one view is used by commands that compare and integrate files. A stream’s owner can customize the template from which the views are generated. For small streams, the default template is not likely to need attention. For large streams, where enormous file trees are involved, the template can maximize code reuse and enforce best practices. (You can bet I’ll be blogging more about this!)

All streams descending from the same mainline are rooted in the same depot. The streams //Ace/MAIN, //Ace/DEV, and //Ace/jebwork, for example, are all rooted in the “Ace” depot.  A stream’s root is always one level below its depot’s root, making a flat and predictable namespace. You can root as many streams as you want beneath the same depot, but we recommend having only mainline per depot. Stream depots are for the most part just like regular Perforce depots. So another way of looking at streams is as subdirectories in a depot tree:

Folders in a stream depot

Here’s what’s different about stream depots: you must use a stream workspace to submit files to them.  A stream workspace “belongs” to a single stream, and gives you the logical view of that stream. With stream workspaces you don’t have to set up a client view; views are automatic. If you switch a workspace from one stream to another, or if you modify a stream’s view template, your workspace view changes on the fly.

In my next blog I’ll give a brief tutorial showing how easy it is to set up and use streams. Meanwhile, I look forward to reading your comments.