April 4, 2013

Task Streams in Perforce 13.1

Healthcare
Streams

New in Perforce 2013.1, task streams are an incredibly useful concept in version management, and Perforce just made them easier to use and more visible. Let's have a look at task streams and how they can help your workflow. Then I'll talk a bit about Perforce's implementation of them.

Function

As the name implies, a task stream (or task branch) is a branch dedicated to a single task on which you're working. Most of the time a task stream is used by a single person, although you might have some other team members involved for review and minor changes as you near completion of the task.

In Perforce's stream graph you'd see a picture like this:

task streams

There are three task streams for three tasks, connected to an upstream branch. That upstream branch is a more stable place where completed tasks are promoted. It could be a mainline, an integration branch, or some other point of collaboration.

Task streams provide a perfect container for a small piece of work, just like streams or branches provide good containers for integration work, release maintenance, and so on. What's so useful about task streams?

  • You can isolate a single task until it's ready to share.
  • You provide a container that's bigger than a single atomic changelist or commit. That means that any changes suggested during a review process can be incorporated into the original body of work.
  • You're more Agile. If you follow the Kanban philosophy, or adhere to one piece flow, then a task stream is perfect. A developer grabs a task, makes a task stream for that task, and works in that task stream until the task is done.
  • If you believe in continuous delivery, then everything that contributes to done can be performed on the task stream. Continuous integration, on-demand test and deployment, static analysis: it's easier to configure these tools to work on a branch rather than a pending commit.

We've seen task streams enjoy a resurgence in popularity over the past few years. They were a popular concept in the ClearCase world, but the management and complexity were a bit much (as with many things in ClearCase). Now a few trends are taking us back in the task streams direction.

Form

Task streams have gotten easier to work with. If you're using git as your local repository, then your git repo is your task branch(es), connected to the upstream Perforce repository through Git Fusion.

If you keep your task streams in the main repository, then Perforce streams have improved the visibility of streams and made the workflow and management a lot simpler. Now the 2013.1 release makes task streams even easier and more efficient.

First, a task stream is a new stream type. It'll be easy to identify task streams in the stream graph.

Task streams are easy to make, just like any other type of stream. But task streams can also work with non-stream branches in Perforce. That means that you can use all the power of a task stream even if you're not using the rest of the streams framework. How? Just create a task stream with no parent, and populate it from a non-streams branch.

Finally, task streams have a little bit of special sauce in the implementation. Most of the metadata about a task stream is private to that stream and doesn't go into the main Perforce database, but are kept in a separate set of private tables. Only the files you actually modify on a task stream are present in the main database. When you're done with the task stream the private data can be unloaded to an unload depot, or removed just by deleting the stream definition. (Unloading the data makes it possible to recover it later.) After you're done, you end up with something that looks a lot like a sparse branch. None of the implementation changes really modify the way you work with a task stream though; if you're hacking away on your code, it walks and talks like a regular stream.

So, while you can work in a task stream transparently, there's much less data kept around permanently. That means you can use a lot of task streams with very minimal overhead. And in my experience, once you get used to task streams, you'll end up using a lot of them.

There are a few guidelines and restrictions around the use of task streams. The release notes have full information, but here are the highlights:

  • We recommend you use the p4 populate command to put data into a task stream. (This command is the best way to put data into any new branch or stream.)
  • Task streams cannot be re-parented. They're tied tightly to the original source.
  • Task streams cannot have children of their own.
  • You can have multiple users (workspaces) using a single task stream.
  • Task streams can be converted to regular streams if they become a more permanent working area.
  • Task streams that are parented to another stream have to follow the merge-down copy-up paradigm.

Try it!

Grab the 2013.1 release of our Software Version Management and give task streams a try! We'd very much like to hear your feedback.