February 21, 2012

New in 2012.1: Virtual Streams


Perforce Release 2012.1 introduces the virtual stream, a powerful abstraction of the stream feature. A virtual stream isn't actually a stream at all -- it is simply a filtered view of its parent stream. Creating and using a virtual stream, however, is the same as creating and using a real stream. I'll demonstrate some of that here, building on the stream view explanation I gave in an earlier blog.

Virtual Streams as View Filters

Let's say you're working at Ace Engineering, where there's an active development branch defined thus:

Stream:  //Ace/dev
Parent: //Ace/main
Type:   development
    share ...

//Ace/dev is a depot path, and it contains files. You can work with these files by switching your client workspace to the //Ace/dev stream. But what if the //Ace/dev path is huge? And what if you have a team of developers who work with only a small subset of these files? Yes, you could branch a smaller child stream from //Ace/dev. But now, with virtual streams, you don't need to. You can create a virtual child stream that simply narrows the //Ace/dev stream's view. For example:

p4 stream -o //Ace/devDocs
Stream:  //Ace/devDocs
Parent: //Ace/dev
Type:   virtual
    share docs/...

When you switch to your workspace to the //Ace/devDocs stream, you get a filtered view of the //Ace/dev path:

p4 workspace -s -S //Ace/devDocs
p4 workspace -o
    //Ace/dev/docs/... //your_workspace/docs/...

And when you switch it back to the //Ace/dev stream, you get the entire view of the //Ace/dev path:

p4 workspace -s -S //Ace/dev
p4 workspace -o
    //Ace/dev/...  //your_workspace/...

When you're switched to a virtual stream, Perforce lets you submit files to its underlying, real stream. So your workspace behaves the same whether you're switched to //Ace/dev or //Ace/devDocs -- in either case, you are working with files in the //Ace/dev path. The only difference is in how much of the //Ace/dev path is in view.

Virtual Streams as Tethers

Another great use of a virtual stream is as a persistent tether for a group of workspaces, or even for a group of streams.

For example, Ace's automated QA system runs tests in three automated client workspaces, QA-MacOS, QA-Windows, and QA-Linux. The QA tests having been running against the //Ace/releaseA stream, but now we want to run them against the new //Ace/releaseB stream. We have a virtual stream configured as the tether for the automated clients:

p4 clients -S //Ace/QAtether
Client QA-Linux (...)
Client QA-MacOS (...)
Client QA-Windows (...)

p4 stream -o //Ace/QAtether
Stream: //Ace/QAtether
Parent: //Ace/releaseA
Type:   virtual

To redeploy testing to the new stream, the //Ace/QAtether stream is reparented from //Ace/releaseA to //Ace/releaseB. (Reparenting can be done by dragging and dropping the //Ace/QAtether stream in P4V, or by editing the QAtether stream spec and replacing the "Parent" field.)

p4 stream -o //Ace/QAtether
Stream: //Ace/QAtether
Parent: //Ace/releaseB
Type:   virtual

The simple act of reparenting redefines all of the test client views at once. The next time the automated tests run, they'll be testing the new //Ace/releaseB stream.

Virtual Streams and the Flow of Change

A virtual stream can have child streams, real or virtual. For instance, Bob can branch his own development stream from the virtual //Ace/devDocs stream.

Branching from a virtual stream is exactly the same as branching from a real stream. Bob can use the new p4 populate command to branch his //Ace/bob stream:

p4 stream -t development -P //Ace/devDocs //Ace/bob
p4 populate -S //Ace/bob -r

Like a real stream, a virtual stream inherits its view from its parent, and passes its inherited view on to its children. A stream's view is used internally to generate two mappings, one being a mapping between the stream and its client workspaces, and the other being a mapping between the stream and its parent. With a virtual stream's view, however, the mapping always refers to the underlying, real stream's path. A virtual stream's nominal path never appears in any mapping.

You may recall that you can use the p4 branch -S command to display the generated stream-to-parent mapping for a stream. Let's take a look at the mapping generated for //Ace/bob:

p4 branch -o -S //Ace/bob dummy
    //Ace/bob/docs/...  //Ace/dev/docs/...

As you can see, the mapping is from one real stream to another; the virtual parent stream's nominal path ("//Ace/devDocs") does not appear. You can tell that the virtual stream's view has been inherited, however, because the mapping is restricted to the "docs" path.

The fact that the parent stream is virtual doesn't change the way p4 merge and p4 copy work. Just remember that your workspace must have a view of the target stream, and that your p4 merge or p4 copy command must refer to the child stream. (This is the case whether virtual streams are involved or not.) Here's how Bob merges the latest changes into his //Ace/bob stream:

p4 workspace -s -S //Ace/bob
p4 merge -S //Ace/bob -r
p4 resolve
p4 submit

And here's how he promotes his //Ace/bob stream to its parent, //Ace/devDocs:

p4 workspace -s -S //Ace/devDocs
p4 copy -S //Ace/bob
p4 submit<

For another demonstration of virtual streams and the flow of change, let's go back to the virtual //Ace/devDocs stream from the first example. Take a look at the branch mapping that is generated from the //Ace/devDocs stream view:

p4 branch -o -S //Ace/devDocs dummy
    //Ace/dev/docs/...  //Ace/main/docs/...

As you can see, this is a useful mapping for integrating files in the "docs" path. For example, here's how we'd use it to promote docs from //Ace/dev to //Ace/main:

p4 workspace -s -S //Ace/main
p4 copy -S //Ace/devDocs
p4 submit

Virtual streams are a fantastic enabler of clever and succinct branching strategies. There's so much more you can use them for than I have room to describe here. I hope you'll them a try as soon you get your hands on the Release 2012.1 beta. If you have questions or observations, let's discuss them in the Streams forum.