A Continuous Deployment Maturity Model
Use a capability maturity model to guide your migration to continuous deployment.
Continuous deployment has a profound impact on engineering performance. It allows you to quickly and safely iterate on software in production, while gaining realtime insights about your operations and your customer needs. As a result, it supports a culture of high ownership and cross-functional collaboration.
But what does the path to continuous deployment look like, and how do you make the transition happen? This post is my attempt to answer those questions.
My team at Cornerstone OnDemand went from a handful of production deployments annually to more than 2,000. We automatically deployed code changes to production as soon as they were merged to trunk, often in under 20 minutes.
But switching to continuous deployment was not an overnight change for us. The team spent over a year building tools, adjusting processes, and establishing the culture required for successful continuous deployment. In the years since then, we continued to develop and refine our practices.
As with any large change, you don’t have to do everything at once. If you break down the process into a series of incremental steps, it’s easier to make progress, and you’ll start to see the benefit of these changes long before you reach the end goal.
So rather than describe what a continuous deployment environment looks like when you’re already doing it, I wanted to share the incremental changes you’ll likely need to implement, across several different areas. There’s a handy tool for just this sort of thing, and it’s called a “capability maturity model”.
Maturity Models
In a nutshell, a maturity model helps you evaluate your current performance in a specific area, and suggests what capabilities you should pick up next to improve.
You traditionally measure performance in five levels: Initial, Repeatable, Defined, Capable, and Optimized. Although if you prefer, you could simply think of them as Level 1 to Level 5. The key thing is that once you’ve identified what level you’re at, you should strive to reach the next level before you attempt to add capabilities from the higher levels. This gives you a blueprint for how to incrementally improve, until you achieve mastery in a specific area:
When it comes to complex initiatives — such as introducing continuous deployment — it’s rarely enough to measure your maturity in a single area. In these situations, you can evaluate maturity across multiple dimensions. It’s helpful to represent this information in a grid, where each row shows a dimension, and the columns indicate levels and the associated capabilities. Here’s an abstract representation of what this might look like:
A maturity model is a tool, not gospel: The idea here is to offer guidance for teams that want to make the transition to continuous deployment, but don’t know what concrete steps to take, or in what order. Depending on your specific situation or organization, your journey may be different — and your destination as well.
A Maturity Model for Continuous Deployment
My team at Cornerstone identified six key areas that we considered crucial for continuous deployment:
Test automation is important because there is no latitude for manual deployment decisions with continuous deployment. Once you merge code it goes straight to production, and you need enough automated safeguards to ensure this will not break existing functionality, or introduce new issues. For more on this, see my previous post on test automation for continuous deployment.
Observability matters because you’re constantly shipping code changes to production, and you need to find out immediately if something went wrong. With good observability tools, you can pick up anomalies in production as they happen, and link them directly to the corresponding code change. This allows you to quickly implement and ship a fix. You should also instrument internal tools to ensure you can measure and troubleshoot every part of the continuous deployment workflow.
Deployment automation is at the core of the continuous deployment journey. Each engineer may be deploying to production several times a day, which means they use the deployment tooling a lot. The tools should be safe, robust, and automated to the point where engineers trust them, but don’t have to spend much time thinking about them.
Engineering process changes allow you to safely deploy smaller increments of work. It’s particularly important to optimize for flow, to minimize the time between code change and production deployment. And much of the benefit of continuous deployment will only be realized when the team measures the customer impact of each change, and uses this information to guide further development.
Release process is a cross-functional concern, in that you need to engage stakeholders across the organization to establish working agreements for continuous deployment. To facilitate this, you need a mechanism to separate code deployments from product releases. If you work in a large company, my earlier post on continuous deployment in the enterprise offers some ideas for how to do this.
Cultural and organizational change is key, but often the hardest to implement. In my experience, the most important change is to develop a culture of ownership, where engineers take end-to-end responsibility for everything they work on. This will be easier in an organization with integrated product management, engineering, QA, and operations teams.
Here’s my suggested continuous deployment maturity model, based on these dimensions:
If you prefer to view or save this in PDF form, here’s a download:
Applying the Model
Whether you use this model or construct your own, there are a few things to keep in mind…
Ideally, you make progress on all these dimensions in parallel. But since this is not always practical, I’ve sorted the dimensions from top to bottom in the order that you should probably start. For example, you’ll benefit immediately from improved test automation and observability — but increased deployment automation will have limited impact if you don’t have the other two.
In this context, you may be surprised to see culture at the bottom. I’ve often heard objections to continuous deployment on the grounds that “we don’t have the right culture for it”. The assumption seems to be that you need to establish the right culture before you can start.
In my experience, this is entirely backwards: You can never build the right culture if your engineers don’t have the tools to safely do their job. Once you build out your automation capabilities, and ratchet up your processes to deploy more frequently, engineers will feel more empowered, and will naturally take on more ownership. Culture change will sprout from this, and you can help it along with some gentle nurturing.
Either way, it’s a good idea to at least achieve the “Initial” state in all dimensions before you start on continuous deployment. Without these fundamentals in place, increased deployment frequency will reduce safety and increase manual deployment work.
Here’s a quick animation of how we progressed through this framework with my team at Cornerstone OnDemand. As you can see, it was not a linear process — and we still had a few areas that needed work:
Approaching Done
Switching to continuous deployment is itself a continuous process. You build up the technical and cultural maturity over time, and continue to fine-tune it indefinitely. That makes it hard to pinpoint the moment when you’ve “arrived”.
For my team at Cornerstone OnDemand, the aha moment came when newly hired engineers were able to deploy a code change to production in their first day on the job. They would merge a change to trunk, watch the build tools automatically validate and deploy it, then test drive it in customer-facing environments just minutes later. This was an incredibly empowering experience for new hires, and brought home the level of automation and safety we had achieved in our deployment tooling.
The maturity model here is intended as a starting point and a guide. But the diversity of engineering organizations guarantees that every journey will be unique. So if you had a different experience introducing continuous deployment, or developed a different continuous deployment maturity model, I’d love to hear about it!
We need an excel version of the maturity matrix, to track progress :D