December 17, 2014

14 Essential Resources for Getting Started and Succeeding with Continuous Delivery

Continuous Delivery

Continuous Delivery (CD) has been an interest of mine for many years. It has been a pleasure to organize the resources and links below.

Continuous Delivery Books and Resources

1. Agile Manifesto

Going back to the beginning, the term CD arises in the first principle of the Agile Manifesto which was developed at a meeting of software development luminaries in February 2001:

Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.

Those luminaries were steeped in practices ranging from Extreme Programming, SCRUM, DSDM to Feature Driven Development.

2. Continuous Delivery

The CD “bible,” by Jez Humble and David Farley (published in 2010) is a very comprehensive guide to a multitude of practices, principles and techniques that are relevant throughout the whole software lifecycle. It covers a range of topics from Configuration Management (“keep everything in version control”) through to testing strategies and managing data through the pipeline. You can read selectively to address the problems you are currently facing, but the whole book is very instructive.

3. Martin Fowler, Kent Beck and the Origins of CI

The practice of Continuous Integration (CI) is a foundation stone for CD, and Martin Fowler’s article, originally written in 2000, is the canonical reference (do browse around his blog while there). It refers to experiences with Kent Beck on the C3 project where many XP (Extreme Programming) practices first evolved. Kent’s book Extreme Programming Explained: Embrace Change from 1999 was an early hit in the agile world.

4. Dynamic Systems Development Model

The DSDM (Dynamic Systems Development Method) was initially developed in 1994. There’s an interesting talk on its founding by Jennifer Stapleton, who became DSDM Technical Architect and with whom I worked at Logica. She describes how competing companies managed to work together on the development of a common method. It’s interesting to see its formalization and how it integrated with other methodologies such as Rational Unified Process (RUP), XP and PRINCE2.

5. Lean Manufacturing and Lean Thinking

Another closely related cornerstone for CD are Lean development principles. The origins are in the Toyota Production System which was first described as “Lean manufacturing” in the 1990s. The book Lean Thinking by James Womack and Daniel Jones (first published in 1996) is an excellent foundation.

6. Lean Software Development

The application of these principles to software development was described by Mary and Tom Poppendieck with their 2003 book Lean Software Development: An Agile Toolkit. This adapted the seven lean principles to the world of software, and popularized techniques such as Value Stream Mapping. It is still well worth reading if you haven’t done so. Equating inventory in the world of manufacturing to the concept of “work in progress” or features that have had time spent on them but aren’t yet being used, is very powerful.

7. Lean Software Development: From Concept to Cash

The Poppendiecks followed up in 2006 with Implementing Lean Software Development: From Concept to Cash – another excellent source of advice and more detailed recommendations, responding to feedback on their earlier volume. I can remember hearing them talk live around that time. These books quote figures from the Standish Group’s 1994 Chaos Report, which suggested that 16% of projects were successful, 53% were challenged and 31% failed. Interestingly, there are some alternative views on the validity of those figures by Scott Ambler as well as this article by J. Laurenz Eveleens and Chris Verhoef.

Another key point which they introduced me to was the importance of getting the requirements right – to quote Peter Drucker “There’s nothing so useless as doing efficiently that which should not be done at all”. The same Standish Group report stated that in their survey over 60% of features delivered were rarely or never used – imagine the waste! From a CD point of view, the business does not always know what is the right thing to develop. However, the ability to release small units of change with a high frequency, allows the business to experiment more rapidly and to evolve towards the right systems and products. See the 2014 State of DevOps report linked towards the end of this article.

8. Lean references

The Poppendiecks have an excellent reference page of books on their site – that should keep you going for a month or two!

9. Brad Appleton and Steve Berczuk – Software Configuration Management Patterns

These days patterns are perhaps a bit “old hat” and almost fading into the background as names that everyone knows and uses (if not always correctly!). The original “Gang of Four” book Design Patterns was a revelation when it came out, and prompted lots of people to look up the origins in Christopher Alexander’s wonderful books.

Brad and Steve worked with Ralph Cabrera and Robert Orenstien to document patterns in SCM branching. That builds upon previous work, for example the paper which Christopher Seiwald and Laura Wingerd of Perforce first presented in 1998.

The book “Software Configuration Management Patterns” is the result of Brad and Steve pulling together the previous  work.

Around 2004 I started writing a monthly column for CM Crossroads together with Brad and Steve. One of the presentations I did for the BCS CMSG in December 2004 was on the Agile Difference for SCM. Brad in particular has an amazing ability to keep track of lots of different references and information, and I can thoroughly recommend his blog article on Lean Software Development Resources, which links to some of the above as well as many other interesting sources.

10. ITIL – the IT Infrastructure Library

Another source of ideas was around work and colleagues in configuration management processes. I worked on a couple of projects with Shirley Lacy who was a co-author of the Service Transition book for ITIL V2, and then Project Mentor for the ITIL Version 3 Update published in 2011. While ITIL processes (and related the ISO/IEC 20000 standard) have made a huge difference to many organizations, they are often ignored those parts of the organization involved in software development. This I think is short sighted, and there are valuable ideas and approaches to be gained from learning about ITIL. An ITIL Foundation Certificate and some of the related Intermediate and Expert level qualifications are one of the relatively few worthwhile qualifications in general IT related processes. There are plenty of jobs which require them. That said, like many other processes, they can be read and applied without the necessary understanding and application to specific circumstances, and some rather ineffective processes which are supposedly “ITIL Compliant” may result. As an entertaining read with an alternative viewpoint to help you avoid some of the pitfalls from Service Management run amok, I can recommend: Introduction to Real ITSM – by Rob England (the IT Skeptic).

11. The Visible Ops Handbook

Shirley advised me to read the Visible Ops Handbook from the IT Process Institute (ITPI), written by Gene Kim and Kevin Behr. This is not perhaps as well-known as it deserves to be. Based on three years of study of high performing organizations, they realized the importance of managing change properly. For example, 80% of outages are self-inflicted by poor change and release management processes, along with low quality of said changes. So a pretty small percentage of outages are actually due to things like hardware failure. The highest ROI for an organization is implementing effective release management processes.

12. The rise of DevOps – and the Phoenix

From the ITPI website we can see the link to another theme closely related to CD – that of DevOps:

The ITPI has long been a proponent and believer in DevOps as an appropriate relationship between development groups and efforts and IT Operations.  The original VisibleOps Methodology was in part DevOps before DevOps was even coined as a method or code of integration between development and operations.

Gene Kim and Kevin Behr have since gone on to co-author The Phoenix Project: a Novel about IT, DevOps and Helping your Business Win, published in 2013. This is both entertaining as a novel, as well as educational in terms of improving your software development processes – along the lines of The Goal by Dr Eliyahu Goldratt.

13. CD and DevOps connections

There is a very strong overlap between CD and DevOps, and there are plenty of good articles and to find on the subject of DevOps. For a good explanation see this history of DevOps by Damon Edwards or this presentation about the experiences at Etsy (note the ITRevolution site run by Gene Kim where both articles are posted). For me DevOps is a subset of CD, though I have seen arguments the other way.

14. The state of DevOps

And to finish this article, check out the Continuous Delivery site, maintained by Jez Humble. A recent article mentions the 2014 State of DevOps Report which is well worth signing up to get a (free) copy. Some of the highlights:

  • Strong IT performance is a competitive advantage         
  • Job satisfaction is the No. 1 predictor of organizational performance
  • Version control is one of the top two practices correlated with Deployment Frequency, Lead Time for Changes and Mean Time to Recover (MTTR)

So, a whistle stop tour to enlighten and entertain you, or at least provide some light bedtime reading – do let me know what I might have missed! What are your favorite references?