April 19, 2011

Streams and The Flow of Change

Streams & Branches

Today, in my series on Perforce streams, I'm going to go over how stream types and stream behaviors can orchestrate the flow of change. Stream support is schedule for Release 2011.1, which will be out later this year.

There are three stream types in Perforce: "mainline", "development", and "release". A mainline is a stream with no parent. Its spec might look like this:

Stream: //Ace/main
Parent: none
Type:   mainline

(Every stream is defined with a spec. I'm showing only the relevant portion of stream specs in these examples.)

Only one type of stream can be an orphan, and that's a mainline. A stream with a parent must be either a development stream, or a release stream. The distinction is in how change is expected to flow -- by merging, or by copying -- between a stream and its parent.

For example, consider this stream:

Stream: //Ace/laura-dev
Parent: //Ace/main
Type:   development

The //Ace/laura-devstream is a child of //Ace/main, and the fact that its type is "development" tells us that Perforce expects it to pull changes from //Ace/mainby merging, and to push content to //Ace/mainby copying.

A development stream and its parent

This is very much how branching works in other version control systems. You could think of development streams as normal branches, used by normal developers, for normal tasks. For a small project, a mainline and a handful of development streams may be all you need.

A "release" stream is the opposite: it pulls from its parent by copying, and pushes to its parent by merging. This may sound odd to you if you haven't read my book or seen any of the Flow of Change presentations we've given. Consider this use case:

Stream: //Ace/stable
Parent: //Ace/main
Type:   release

Here we have //Ace/stable, a release stream whose parent is the mainline. We've branched it because we need a place to test and stabilize our next release. At the same time, we don't want to keep developers from promoting their work on future releases into the mainline. We do our system testing and bug fixing in the release stream, and as we fix bugs, we merge the fixes into the mainline.

And let's say our product is a web-hosted product. When the //Ace/stablestream is ready to ship, we label it and publish it to our live content distribution site. This frees the //Ace/stablestream for stabilizing the next release candidate, which we promote (copy, that is) from //Ace/main. And so on.  

Unlike development streams, which are typically less stable than their parents, a release stream is more stable than its parent. A convention I started using in my book was to show stability on a vertical scale, from low to high. The flow between //Ace/stableand //Ace/main, then, looks like this:

Just think "merge down, copy up". If you'll take a look back at this post, you'll see a diagram showing a mix of release streams and development streams. The release streams are high on the diagram, and the development streams are low. As you can see, "merge down, copy up" applies to all the streams in this system.

(An aside: I used to talk of streams being "firmer" or "softer" than their parents, but outside of the people who'd read my book, no one knew what I meant. Now I say "more stable" and "less stable", though that's kind of a mouthful. In writing this post, however, it hits me that the opposite of "stable", with regards to the flow of change, may be "agile". I'll explain why in a future post.)

So, a release stream expects promotions from its parent, and a development stream expects merges from its parent. What do we mean by "expects"? We mean the way the new copyand mergecommands will work.

With the advent of streams, many Perforce commands now sport the -Sflag, which tells them to operate on a stream. The mergeand copycommands are special. Not only do they allow the -Sflag, they check the expected flow of the stream with respect to its parent. For example:

p4 merge -S //Ace/stable

This command merges changes from //Ace/stableto its parent, //Ace/main. Since //Ace/stableis a release stream, Perforce does indeed expect change to flow to its parent by merging, so it allows the operation. But when we try merging in the reverse direction, we get an error:

p4 merge -S //Ace/stable -r
Stream //Ace/stable needs 'copy' not 'merge' in this direction.

In other words, Perforce does not expect to do any merging from //Ace/mainto //Ace/stable. Copying in this direction is expected, however, so the copycommand does work:

p4 copy -S //Ace/stable -r

The mergecommand is new in the upcoming 2011.1 release. The copy command was introduced in 2010.2, and has been retooled for streams. The -Sflag tells these commands to operate on a stream and its parent, the default direction of flow being toward the parent. The -rflag reverses the flow.

A stream's type controls how change flows between it and its parent. There's also a way to control whether change flows between a stream and its parent. I didn't show it in the previous examples, but stream specs have an Options field that fine-tunes the flow of change. Here's what Options looks like in the spec for the //Ace/stablestream:

Stream: //Ace/stable
Parent: //Ace/main
Type:   release
Options: fromparent toparent allsubmit locked

The "fromParent" and "toParent" toggles control whether parent-to-stream and stream-to-parent flow are expected at all. In this case, change is expected to flow in both directions -- the default behavior. But if the owner of this stream changes "fromparent" to "nofromparent", for example, the copycommand will refuse to copy from //Ace/mainto //Ace/stable. It's not completely forbidden, however -- you can always force Perforce to go against the flow. (There's a flag for that.)

Which brings me to the "allsubmit" option -- the stream's owner can change that to "ownersubmit" to prevent anyone but herself from submitting changes to her stream. The "locked" option is also important; it prevents other users from modifying the stream spec. The combination gives owners control over their own streams; other, ordinary users can't thwart it. Content may be pulled from an "ownersubmit" stream into other streams, but the only way changes can get into the stream are through the actions of its owner.

As you can imagine, all of this together presents some nice possibilities for guiding the flow of change in and out of streams. If it sounds complicated, fear not -- almost everything you need to do with streams can be done, in most cases, using the default behaviors. For all the special and extreme cases that come up in enterprise software production, the configurable behaviors are standing by to handle them.

In my next post I'm going to tackle stream views -- how they work, and how you can use them to make big branches with small footprints. Until then, I look forward to your comments.