Understanding Configuration Management
in Today's Market

[ Deutsch | Français ]

Few things get the blood pumping like successfully debugging a block of code - and then finding out that someone else debugged that block yesterday. Is there a bigger adrenaline rush than finishing a large software project - after the scheduled ship date has passed? What's more invigorating than hunting for the most recent revision of that software module, completed three weeks ago by the team, that has since been reassigned?

All of these problems could be avoided by employing software configuration management (SCM). SCM tools are the means by which the evolution of a software product can be tracked and managed. As a subject, SCM has often been accused of lacking excitement. That is, in fact, the point of SCM: some excitement can be done without.

Large software organizations - especially those running multiple concurrent development projects, with each project divided among teams - are beginning to agree. These companies have been scorched often enough to have a keen understanding of how critical it is to efficiently manage the software development process. They are also among the first to learn the hard lesson that it is difficult to implement a truly effective SCM system on your own.

The genesis of SCM

In the past, before SCM was understood to be a task unto itself, development organizations trusted that programmer activity would be managed as a logical outgrowth of the software development process - that the development teams would keep track of software evolution on their own. After it became apparent that this was at best inadequate and at worst no solution at all, some software companies cobbled together their own SCM tools. Some of these homegrown tools have evolved into open source projects. Alternatively, some companies turned to document management systems for their SCM functions.

But in the last few years, with many software projects running into many millions of lines of code, with multiple teams working on separate modules of a program and with teams geographically dispersed, SCM became too large and important a task to be performed by anything less than dedicated, professional-level, commercial tools. Building your own tools is now impractical, open-source software is now too limited, and document management systems have always been unsuited for the task. In response to these pressures, a commercial market for SCM tools began to emerge, starting in the early 1990s.

The benefits of professional SCM tools

Among the tangible benefits of SCM tools are the ability to:

• create a more predictable software development and release cycle
• automate repetitive development tasks
• manage the concurrent development of multiple codelines, which brings the benefit of increased productivity
• deliver a more bug-free product
• create distributed development teams regardless of their geographical location

SCM may have a reputation for being about as interesting as accountancy, but the consequences of a casual approach to SCM are no less dire than the consequences of sloppy bookkeeping. That's why the smartest and most competitive software development organizations are the most enthusiastic adopters and users of formal SCM systems.

The market

Companies that have embraced the SCM approach wholeheartedly include Silicon Graphics, IBM, PowerSoft, and German software giant SAP AG. And if there is any doubt about the value of SCM, the numbers should dispel it. SCM tool revenues have been increasing at an average rate of approximately 50 percent a year since 1995, according to Ovum Associates. In 1999, the global market for what Ovum calls "configuration management" or "CM" tools and services had grown to nearly $1.5 billion.

"A principal source of growth has been that users have seen successful results in early trials, and have started applying CM tools to other projects throughout the organization. Once the benefits of CM are experienced, they are not abandoned lightly," according to Ovum's recent report on the market.

SCM tools

The typical way to categorize most software products, SCM products included, is to place them on a high-end/low-end scale.

At the low-end, no cost, open-source software SCM tools are available. Some of them can be adequate, though they are always limited. Eventually, however, without modification, they typically fail to meet the real demands of a full software development cycle. The ability to modify these tools implies finding and maintaining people to do the job; and few software development organizations are willing to support that overhead. The low end of the commercial market is occupied by a number of tools, the majority of which are fundamentally simple version-control tools. Usually a graphical user interface (GUI) sitting on top of an open-source software tool.

High-end SCM tools offer a greater array of features, typically providing more options for tracking, creating reports, and generating analytical data. They are often highly configurable. If you look at the product descriptions, there appears to be a great deal of differentiation among SCM tools, with a range of prices and features offered in bewildering combinations.

Real distinctions among SCM tools

Comparing product data sheets can be misleading. A more useful, but more difficult, means of distinguishing the value of one SCM tool against the other is to evaluate how well they actually perform. By this measure, the distinctions among SCM tools shake out in a way that has little relationship either to their placement on the high-end/low-end scale or to price.

A common drawback among many of the products in the low range is an inability to address the needs of sizeable, complex software development projects. High-end SCM tools can actually be worse. In practice, many features and configurability means that developers are obliged to engage in a substantial amount of customization to make high-end SCM tools functional. Tools in this category are often cumbersome to maintain, and have hidden costs. Two of the more expensive hidden costs are the necessity to hire specialized administrators, and investments in additional computing resources.

The features that are at first glance so appealing are indeed useful - to managers and administrators. But the accretion of more tracking, more reports, more data, more management, and more administration eventually becomes an unacceptable burden to programmers and programming teams. The dirty little secret of SCM is that in more cases than anyone cares to admit, software engineers have found some SCM tools so burdensome they refuse to use them. These tools are of limited - if any - value.

If not for developer resistance to using cumbersome SCM tools, the SCM tools business could be much bigger today than it is. That it is a $1.5 billion market is testament to how desperately software development organizations believe in the concept and desire tools that work.

The question that most (but not all) SCM tool vendors fail to ask themselves is, what's more important - the paperwork or the product? In other words, when SCM tool vendors create their products, do they focus on the SC or the M? The question is far from frivolous; the ramifications are real and profound. If engineers won't use the tools or won't use them properly because the M - the management elements - are too cumbersome, then an investment in SCM tools is counter-productive and wasteful. However, if the tool's emphasis is on the SC - the software configuration elements - then the emphasis is on developer productivity, and the resistance to using them melts away. Perforce calls tools that emphasize the M in SCM, "process-focused," and the tools that emphasize the SC, "developer-focused."

Process-focused SCMs are built to address the concerns of managers and project administrators. The goal being served is organizational control. Process-focused SCM solutions are highly attractive to highly regulated industries that demand strict development guidelines and watchful management oversight. The key selling point of process-focused SCM tools is often that they represent a turnkey software development solution. Follow the process-focused path, however, and you inevitably end up with SCM tools that are pretty, cheap, and unreliable, or bloated and unjustifiably expensive.

Developer-focused SCMs are built to address the concerns of the application developers. The emphasis here is on making programmers (and programming teams) more productive while also satisfying the requirements of managers and the organization as a whole. Follow the developer-focused path, and you end up with tools that install quickly, require minimal overhead and support, and are unobtrusive as far as the average programmer or software development team is concerned.

Perforce Software prefers this distinction for an obvious reason: Perforce is one of the few SCM tool vendors that follows the developer-focused model, and Perforce SCM tools perform well and are adopted with enthusiasm.

It is also true that Perforce doesn't fit well into the high-end/low-end scale. The company's SCM tools are inexpensive, yet perform as well or better than so-called high-end - read "expensive" - tools, fully serving the needs of both software developers and their managers. Furthermore, Perforce's SCM tools do not have the same volume of features that some other "high-end" SCM tools do. This is intentional. The company purposefully designed its tools to be lightweight - occupying few system resources, requiring little overhead, and performing the necessary tracking and management functions without loading up on potentially useful but ultimately unnecessary features. The result is better performance.

If performance is to become the standard for evaluating SCM tools, the criteria that are most relevant are usability, scalability, strength of the model, and reliability.

• Usability - Is the system fast enough? Is it fast enough over various networks? Does it tolerate a heterogeneous computing environment? Is using the system burdensome to a developer?
• Scalability - What happens when you do load your 20,000 source files (or 400,000 Web pages) into the SCM system? Do you still have a system? Or did you have to break it up into 20 separate ones?
• Strength of Model - Can the SCM system manage multiple, concurrent development and release codelines? Does it keep track of what bug fixes were made, where?
• Reliability - Will the tools work as advertised, and will they actually be used because of it? How do the tools rate in terms of quality of service (QoS)?

By these measures, Perforce SCM tools perform extremely well. They are the benchmark in usability, they scale extremely well, they are tremendously reliable, and the underlying model is strong.

Summary

SCM tools are not all alike; they have different approaches to the job, and some are more powerful, useful, and flexible than others. Some SCM tools look sophisticated because they have nice interfaces, but their GUIs are the most sophisticated thing about them. Some SCM tools fit into any process flow, while others force engineers to adapt their flow to fit the tool. Some, such as Perforce tools, are powerful, flexible, fit into almost any process flow, and are so easy to use that some developers have reported getting up and running in as little as 20 minutes. Some SCM tools actually perform.

 1. For more information, read the article by analyst firm Quocirca:
     Through the fog... Software configuration management [PDF]