October 7, 2015

Configuring TeamCity with Perforce Streams

Continuous Integration
Streams & Branches

When JetBrains started developing TeamCity — a Continuous Integration (CI) server — about eight years ago, other projects of ours, such as IntelliJ IDEA, had recently moved to Perforce. Naturally, the TeamCity project used Perforce right from the start.

Since one of our goals has always been to provide an ideal CI platform for our own projects, TeamCity needed excellent Perforce support. We started with the fundamentals, like checking out source code and detecting changes that should be built. Then we started adding more sophisticated features like remote runVCS labeling, the difference viewer, and more. Check out the video on the complete TeamCity integration with Perforce to learn more.

Development Process

We’ve always used Perforce branches during development. We have a trunk branch and release branches for all public releases. We tend not to use feature branches though and prefer to use feature toggles instead.

But this approach is not applied across all the code the team maintains. As well as the main product source code, there are also lots of plugins for TeamCity maintained by the development team. The code for many of these plugins resides in Git and Subversion, because we want plugin code to be public. GitHub makes this easy. So, while the core part of TeamCity is developed with the feature toggles approach, plugins hosted in Git often use feature branches. Fortunately, these feature branches are short-lived. Because the plugin code base is usually small and maintained by a single developer, we rarely see merge problems.

Why Switch to Streams?

We recently released TeamCity 9.1, so it was time to create another branch in Perforce. People on the team were used to working with feature branches in Git and wanted this feature in our main code. We stopped and thought, maybe now would be a good time to try Perforce streams? We had heard that Perforce streams let you create branches easily, and that you can create lightweight task streams.

Making the switch seemed low-risk. Our important change history would be preserved. And since streams are just another view on branches, working with streams would be similar for our developers.

Making the Switch

We use the TeamCity product to build the TeamCity code, so although the product already fully supported streams, we double-checked that everything we needed was in place for our own builds. We also needed to check that streams work well in IntelliJ IDEA.

Everything looked promising, so we started the migration. But things did not quite go according to plan!

We spent about a day moving our source code to streams. During this process, we realized that we could take this opportunity to repay some technical debt and rename some of our directories. The result was that we ended up having to repeat the move after we’d made the changes. For anyone else considering this procedure, make sure you’re happy with your directory structure before moving ahead!

Three Considerations When Moving to Streams

1. How Will Moving to Streams Impact Our Workflow?

Moving to streams was not just a technical procedure; it also meant we needed to define a simple workflow. The technical creation of streams went very smoothly, but when we tried to define the workflow, we realized that some concepts of streams were not very convenient .

The most complicated part for us was merging. Perforce streams impose some constraints on the way you work with them. For instance, when you define a release stream with the trunk as a parent, there are two options for merging:

  • To parent: merges from the release stream to the parent (trunk) stream
  • From parent: copies from the parent (trunk) stream to the release stream

Our developers work differently. They can make changes in trunk and then merge them into the release branch , or they can make changes in the release branch and merge them into trunk. In fact, not all of the changes made in the release branch are merged into trunk. There are some files that always differ between the release branch and trunk—for instance, a file with the version of the product. 

It looked like our previous workflow would become more complicated with streams. Fortunately, since streams are a thin layer over traditional branches in Perforce, we were able to create branch mappings and use our usual approach to integrate changes.

2. How Could We Synchronize Between Multiple VCS Roots?

We wanted to use a single workspace and switch quickly between streams. This feature is great as it gives you really fast switching between streams.

But we didn’t anticipate one problem with our project: the plugin code doesn’t live in Perforce. I mentioned earlier that our plugin code is in Git and Subversion repositories. This code is fetched into a project subdirectory via scripts in the root of the core project code. These plugins depend on the API defined by the core project code (i.e., the code in Perforce). So, if someone renames a method in the API, both the Perforce code and the plugin code can be affected.

Now imagine that someone switches to the trunk stream but forgets to switch the subdirectory containing the plugins. If they change API code, this change will affect plugins as well, but not the correct plugin code. Since the plugin code in the workspace corresponds to some release stream that is not the trunk stream, if the developer checks in this change they’ll break compilation in both streams, release and trunk. 

This situation could very easily occur and was therefore very dangerous. The solution was to use two workspaces: one for trunk and another for the latest release. This, of course, was not an issue with Perforce but rather with the way our project was configured.

3. How Could We Fix an IntelliJ IDEA Issue?

At last, we were up and running with streams and able to get on with writing code for TeamCity product features. We use IntelliJ IDEA for most of our development, and initially everything worked fine: commit, update, annotate. However, after a while we discovered an annoying problem. In IntelliJ you can see the history for a file:


And see changed files in a changelist: 


This worked fine for all new commits made into the stream, but not for commits made into our old branches before the migration; it only showed an empty list of files. Everything worked as expected in P4V, so it was clear that the problem was in IntelliJ IDEA. We were able to describe our case to the IDEA developers and they fixed this quickly.

What We Learned

The switch to Perforce streams has provided us with some important insights. We better understand the constraints in our project. For example, the problem we solved by using two workspaces prevents us from using feature branches effectively. We’ve also shown that by consuming our own products, we have a better understanding of our customer use cases as they configure TeamCity with Perforce streams. 

Ultimately, using Perforce has streamlined and facilitated the TeamCity development process.