Dependency Management with Maven
This is the first post in a three part series about managing dependencies with Maven.
The product you work on is built of over 50 components, produced by several teams working on different schedules. Some of these components are more or less aligned on your product schedule, while others are produced almost independently and you treat them like an external library. You need to make sure you have the right versions of all of these components in your workspace in order to develop, build, test, and release your product.
Sound familiar? Dependency management is one of the most difficult parts of developing software and related products. There are a few ways to handle this task, depending on your needs.
Managing Dependencies with Views
Many SCM tools offer some sort of views: selecting which pieces of the repository you want in your workspace. In some cases, views are assembled using concepts like sub-modules or sub-projects, which import data from somewhere else into your workspace. These concepts are useful, in that they let you define these imports and version them. However, they're not always easy to set up and share, so they don't scale very well.
Other systems let you define views using text-based configuration files. These offer you a lot of flexibility, but the ones I've used weren't versioned natively, which made them fragile.
Perforce offers very powerful views in workspaces, branches, and now streams. Views in workspaces and branches give you a lot of flexibility to select the files you need in your workspace, and streams let you define and manage views centrally. (Stream views aren't quite as flexible as traditional views, but they cover the essential cases.)
Even in Perforce, however, relying strictly on views isn't going to work for everyone. In the most complex cases, when you have to frequently select over 50 components to include in a workspace, you need to look at dedicated dependency management systems.
Roll Your Own
Of course, it's possible to build on Perforce views to construct your own dependency management system. When I was on the consulting team at Perforce, I saw and helped build a few of these. While they differ in the details, a basic outline looks like this:
- Store the project's composition, a list of dependencies and other settings, in a text file that is checked in and fully versioned.
- Provide tools for the end user that use the project composition file to build workspaces and assist other operations. These might be scripts deployed on workstations, scripts built into the Perforce broker, or other tools.
- When the project composition changes, notify users to update their workspaces accordingly, do it for them via a script, or rely on Perforce streams to take care of that.
Building your own framework can be quite successful, but it does imply a certain amount of overhead and maintenance.
Maven 2.X is a software project management tool with strong dependency management features. (Other similar tools exist, but I'm using Maven as an example because I've used it for years.)
Maven can be used to construct a project archetype, which defines the structure of a project. For example, Atlassian provides a series of archetypes that lay out the framework of a project to build plugins for tools like Jira. Once you make a new project for a Jira plugin, for example, you get a directory set up properly to hold Java source code, resource files, and test classes.
But you get a whole lot more. You get build targets configured for compiling, packaging, testing, and deploying a plugin into a running Jira instance. And most importantly for our discussion today, you get all of the software dependencies necessary to build a plugin for a specific version of Jira. When you run a build, all of these dependencies will be downloaded into your local Maven repository, and automatically included in your build.
When Atlassian releases a new version of Jira, I'd usually want to recompile my plugins to make sure they still work. I'd simply update my Maven POM (Project Object Model file) to indicate that I want to build against that new version of Jira, and it would go and grab any new or updated dependencies for me.
Working with Dependencies
What does defining a dependency in Maven look like? Here's an example where I want to pull in log4j, a popular Java logging library, into my project.
Later if I decide that I need version 1.2.14 of log4j, I just update the version in the dependency definition.
The POM and Perforce
Since the POM is just a text file, it can be checked into Perforce and strongly versioned. In other words, if your builds start to break, you can use tools like Time-Lapse View on the POM to see if your dependencies are wrong. In very strict environments, you can also put special access controls on the POM to prevent anyone other than product architects from modifying it. Any changes to the POM are, of course, available to anyone working in that project.
Working with Maven and Perforce is also easy, thanks to our handy P4Maven project.
Working with Dependency Source
Normally when using Maven you consume dependencies as packaged libraries, like Java jar files. You can also indicate that you want the source of those dependencies, in case you need to do some debugging. I'm assuming here that you don't need to modify the source of any of the dependencies; a project has to be reasonably modular to take full advantage of Maven.
Now, Atlassian did a fair bit of work behind the scenes to make this so easy. They publish the Maven archetypes as well as base Maven POMs that define those dependencies. Yes, the POM supports inheritance, which makes it easy to centrally define dependencies for a set of related projects.
If you end up using Maven internally, you can easily set up and maintain your own Maven repository. That lets your teams publish their components to your own repository, where they can be consumed by other teams. It's quite easy to use version strings to indicate the stability level of a published component. For example, you could start with 1.0-ALPHA, proceed to 1.0-BETA, and then just to 1.0.
Consult an Expert
Perforce partners like Big Lever provide sophisticated tools and services to help with dependency management, code re-use, and improving product lifecycle efficiency. Of course, our own consulting team can help as well.
If you're new to the game of building products with very complex dependency trees, don't try to go at it alone. We're here to help, so contact Perforce Consulting or Support for pointers. Meanwhile, I'll be following up with another article showing Maven in action.