Configuring TeamCity with Perforce Streams
When we at JetBrains started developing TeamCity—a Continuous Integration (CI) server—about eight years ago, other projects of ours such as IntelliJ IDEA had recently been moved to Perforce. So naturally, the TeamCity project used Perforce right from the start.
Since one of our goals has always been to provide an ideal CI platform for our own projects, the TeamCity product needed excellent support for Perforce. We started with fundamental things like checking out source code and detecting which changes should be built. Then we started adding more sophisticated features like remote run, VCS labeling, the difference viewer, and more. Now the TeamCity integration with Perforce was pretty complete—this video covers some of the features provided by Perforce:
We’ve always used Perforce branches during development; we have a trunk branch and release branches for all public releases. We tend not to use feature branches, though; we prefer to use feature toggles instead.
But this approach is not applied across all the code the team maintains. As well as the main product source code, there are also lots of plugins for TeamCity maintained by the development team. The code for many of these plugins resides in Git and Subversion, because we want plugin code to be public and GitHub makes this easy. So while the core part of TeamCity is developed with the feature toggles approach, plugins hosted in Git often use feature branches. Fortunately, these feature branches are very short-lived—because the plugin code base is usually small and maintained by a single developer—so we rarely see merge problems.
Why Switch to Streams?
Just recently we released TeamCity 9.1, so it was time to create another branch in Perforce. But given that there are people on the team who are used to working with feature branches in Git, and who would like to use that approach in the main code as well, we stopped and thought: maybe now would be a good time to try Perforce streams? We had heard that Perforce streams let you create branches very easily, and that you can create lightweight task streams.
Making the switch seemed fairly low-risk. History—the most important part—would be preserved, and since streams are just another view on branches, working with streams would be similar to working with branches.
Making the Switch
We use the TeamCity product to build the TeamCity code, so although the product already fully supported streams, we double-checked that everything we needed was in place for our own builds. We also needed to check that streams work well in IntelliJ IDEA.
Everything looked promising so we started the migration. However, things did not go quite according to plan!
We spent about a day moving our source code to streams. During this process, we realized that we could take this opportunity to repay some technical debt and rename some of our directories. The result was that we ended up having to repeat the move after we’d made the changes. For anyone else considering this procedure, we suggest you make sure you’re happy with your directory structure before moving ahead!
Three Considerations When Moving to Streams
1. How Does This Impact Our Workflow?
Moving to streams was not just a technical procedure; it also meant we needed to define a simple workflow. The technical creation of streams went very smoothly, but when we tried to define the workflow we realized that some concepts of streams were not very convenient for us.
The most complicated part for us was merging. Perforce streams impose some constraints on the way you work with them. For instance, when you define a release stream with the trunk as a parent, there are two options for merging:
To parent: merges from the release stream to the parent (trunk) stream
From parent: copies from the parent (trunk) stream to the release stream
Our developers work differently: they can make changes in trunk and then merge them into the release branch if everything is OK, or they can make changes in the release branch and merge them into trunk. In fact, not all of the changes made in the release branch are merged into trunk, because there are some files that always differ between the release branch and trunk—for instance, a file with the version of the product.
It looked like our previous workflow would become more complicated with streams. Fortunately, since streams are a thin layer over traditional branches in Perforce, we were able to create branch mappings and use our usual approach to integrate changes.
2. Synchronization Between Multiple VCS Roots
We wanted to use a single workspace and switch quickly between streams. This feature is great as it gives you really fast switching between streams.
But we didn’t anticipate one problem with our project: the plugin code doesn’t live in Perforce. I mentioned earlier that our plugin code is in Git and Subversion repositories; this code is fetched into a project subdirectory via scripts in the root of the core project code. These plugins depend on the API defined by the core project code (i.e. the code in Perforce). So if someone renames a method in the API, both the Perforce code and the plugin code can be affected. Now imagine that someone switches to the trunk stream but forgets to switch the subdirectory containing the plugins. If they change API code, this change will affect plugins as well, but not the correct plugin code. Since the plugin code in the workspace corresponds to some release stream that is not the trunk stream, if the developer checks in this change they’ll break compilation in both streams, release and trunk.
This situation could very easily occur and was therefore very dangerous. The solution was to use two workspaces: one for trunk and another for the latest release. This, of course, was not an issue with Perforce but rather with the way our project was configured.
3. IntelliJ IDEA Support
At last, we were up and running with streams and able to get on with writing code for TeamCity product features. We use IntelliJ IDEA for most of our development, and initially everything worked fine: commit, update, annotate. However, after a while we discovered an annoying problem. In IntelliJ you can see the history for a file:
And see changed files in a changelist:
This worked fine for all new commits made into the stream, but not for commits made into our old branches before the migration; it only showed an empty list of files. Everything worked as expected in P4V, so it was clear that the problem was in IntelliJ IDEA. We were able to describe our case to the IDEA developers and they fixed this quickly; the fix is a part of IntelliJ IDEA 15 EAP and will be in 14.1.5.
The switch to Perforce streams has provided us with some important insights. We now understand better what constraints exist in our project; for example, the problem we solved by using two workspaces prevents us from using feature branches effectively. We’ve also shown that by dogfooding our own products, we have a much better understanding of our customer use cases as they configure TeamCity with Perforce streams.
Ultimately, though, using Perforce has streamlined and facilitated the TeamCity development process. Why not try it yourself and see what it can do for you?