July 19, 2011

The Ongoing Relevance of Continuous Integration


Every 18 months or so, I find myself returning to and rereading Martin Fowler's essay on Continuous Integration. I've been rereading this essay for more than a decade now, since Fowler's first version was published in the year 2000. What Fowler's essay may lack in artistic qualities it more than makes up for with seat-of-the-pants hard-earned wisdom:

Continuous Integrations doesn't get rid of bugs, but it does make them dramatically easier to find and remove. In this respect it's rather like self-testing code. If you introduce a bug and detect it quickly it's far easier to get rid of. Since you've only changed a small bit of the system, you don't have far to look. Since that bit of the system is the bit you just worked with, it's fresh in your memory.

Fowler breaks down the practice of Continuous Integration into the following techniques:

  • Maintain a Single Source Repository
  • Automate the Build
  • Make Your Build Self-Testing
  • Everyone Commits To the Mainline Every Day
  • Every Commit Should Build the Mainline on an Integration Machine
  • Keep the Build Fast
  • Test in a Clone of the Production Environment
  • Make it Easy for Anyone to Get the Latest Executable
  • Everyone can see what's happening
  • Automate Deployment

Of these, my personal favorite is Everyone can see what's happening; as Fowler says, "Continuous Integration is all about communication, so you want to ensure that everyone can easily see the state of the system and the changes that have been made to it." When teams are working together to build software, communication is the hardest part; our development tools and processes need to do everything possible to improve team communications.

Of course, Fowler isn't the only person who's recognized the value of Continuous Integration. Around the same time, Joel Spolsky first published his famous essay: The Joel Test: 12 Steps to Better Code, and its companion essay: Daily Builds Are Your Friend. And at roughly the same time, Ward Cunningham, Ron Jeffries, Kent Beck and the Extreme Programming school were talking about the need for continuous integration:

Extreme Programming is about team responsibility for all code, for coding in a consistent pattern so that everyone can read everyone’s code, about keeping the system running and integrated all the time.

And there have been many others, from Steve Berczuk to Alistair Cockburn to Steve McConnell, who have been making the same points. Nowadays, when these principles are so deeply and broadly accepted, it's somewhat hard to remember that there was a time when these statements were viewed with shock and disbelief: "everyone can read everyone's code"? Egad!

At Perforce, we're deeply familiar with these concepts; our founders have been emphasizing these principles ever since the company was founded. We've seen these techniques work; we use them everyday; our products and our company exist to help you make it possible in your organization, too!

You're surely wondering why I'm droning on with these history lessons? Well, I am a geezer, after all, so perhaps it's no surprise. But really, what disturbs me nowadays is that there seems to be an increasing opinion that Continuous Integration is just about running automated tests. You can find article after article after article about tools and products for running your tests, completely missing the point with perspectives such as

Continuous Integration is a fancy term for “run your project’s tests after someone pushes to the repository and notify interested parties if they fail.”

You'll even find web comics making the point!

All of these writers are well meaning and well intentioned, but I feel they are focusing far too much on the testing angle. Continuous Integration is about far more than just running your automated tests. Let's remember what the Agile Manifesto has to say about this:

Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.
Welcome changing requirements, even late in development. Agile processes harness change for the customer's competitive advantage.
Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.

Continuous Integration is about agility, not about test automation. It's about communication, and about feedback, and about interaction between developers and users. Here's Fowler again:

Frequent deployment is valuable because it allows your users to get new features more rapidly, to give more rapid feedback on those features, and generally become more collaborative in the development cycle. This helps break down the barriers between customers and development - barriers which I believe are the biggest barriers to successful software development.

Happily, there are at least some current writers who seem to be getting the point: here's a nice example by someone who understands that tools are just tools; what's important is helping your team become a team.

Say, has it been more than 18 months since you last read Fowler's essay? Pour yourself a cup of coffee, pull up a chair, and refresh your memory. I can guarantee you'll walk away with a list of improvements you can make in your own organization.