Today's article comes from the IEEE Access journal. The authors are Rajab et al., from Syrian Virtual University. In this paper the authors explore how DevOps teams are attempting to embrace the Capability Maturity Model.
DOI: 10.1109/ACCESS.2025.3542630
The year was 1986, and the Department of Defense had a problem. They were interviewing Software Development agencies in order to award contracts, and they needed a way to pick a winner. All the dev-shops interviewed quite well. They all had expansive portfolios. They’d all write very convincing proposals, they’d give impressive presentations, and they all seemed to know what they were doing.
The issue was, the Department had been fooled in the past. Previous contractors had overstated their abilities. And many, it turned out, weren't actually capable of even a fraction of their claims. Given a contract, they’d try! They’d certainly try!...to build what was asked for, but they wouldn’t succeed. Instead, they’d miss deadlines, or deliver buggy applications, or just give up and abandon the project.
So, the Feds were left with an interesting question: How do you evaluate a Software Engineering team that you just met? How do you spend just a few minutes (or hours) with them, or give them a survey, and determine from that little bit of information what the team is actually capable of?
The result was the Capability Maturity Model (CMM). It defined five levels: from Initial to Optimized. Teams at the low end are chaotic and unpredictable, and at the high end are highly structured and continuously improving. There’s room in the model for everything in between.
Since its inception, the CMM has evolved and been adapted. There are now maturity models for IT service management and cybersecurity, project management, business processes, and many others.
In today’s paper, the authors are looking at how the CMM has grown and evolved in reference to the DevOps movement. That is: how it’s being used to assess DevOps maturity, as opposed to the maturity of the engineering organization as a whole.
To understand this study, we need to start with the original CMM itself and get into the weeds a bit. Then we’ll turn to the authors' analysis and see what they uncovered about the modern usage patterns of this framework.
The original CMM was introduced as a structured way to assess and improve the effectiveness of software development teams. Each of its five levels represent a distinct stage of organizational capability. These levels (1-5) serve as a guide for organizations to identify weaknesses, implement process improvements, and ultimately achieve higher efficiency, higher product quality, and more predictable delivery timelines. The five levels are:
At the lowest level, Initial, software development processes are largely chaotic and unpredictable. Organizations at this level lack standardized procedures, relying instead on the skills and the “individual heroics” of team members to get things done. Projects tend to be reactionary, often suffering from missed deadlines, cost overruns, and inconsistent quality. When problems arise, solutions are improvised rather than guided by structured processes. Since there is little to no documentation, knowledge remains fragmented, making it difficult to replicate success or learn from past failures.
Above this, the Repeatable level represents an environment where some order has been imposed. Organizations at this level have recognized the need for process control and have implemented basic project management practices. While individual projects may follow repeatable procedures, these processes are not necessarily standardized across the entire organization. Managers begin tracking costs, schedules, and progress, allowing for more predictable outcomes. However, the effectiveness of these processes depends on how rigorously they are followed by teams. Some variability remains, as organizations may still rely on individual expertise rather than institutionalized best practices.
The transition to the Defined level marks a significant shift in maturity. At this stage, an organization has established a set of standardized and documented processes that are followed consistently across all projects. These processes are tailored to the company’s specific needs and are designed to ensure repeatability and efficiency. Unlike the previous level, where process adherence may vary from team to team, organizations at this stage enforce company-wide standards. New employees receive training in these established processes, reducing dependency on individual knowledge and improving scalability. The focus shifts from merely managing projects to creating a stable foundation for long-term growth and process improvement.
Next, the Capable/Managed level introduces a data-driven approach to process control. Organizations operating at this level systematically collect and analyze performance metrics to identify areas for refinement. Development practices are monitored using KPIs and quality assessments. These metrics are not only tracked but actively used to make informed decisions. Predictive analytics help identify risks before they become critical issues, allowing teams to proactively adjust their workflows. The emphasis on measurement ensures that process improvements are not driven by intuition or anecdotal evidence but by empirical data.
The highest level of the CMM, Optimizing/Efficient, represents a state of continuous improvement. Organizations at this stage do not merely react to inefficiencies but actively seek out opportunities to enhance their software development processes. They embrace innovation, incorporating new technologies and methodologies to refine their workflows. Process adjustments are not sporadic but built into the company’s culture, ensuring sustained progress. The emphasis on continuous learning and adaptation enables organizations to maintain a competitive edge and respond effectively to changing demands.
And that’s the whole thing, that’s the CMM. And as a former Engineering Manager myself, I can tell you it’s an incredibly effective rubric that you can use to figure out where your team is at. This is especially true when you join a new organization and you’re trying to decide what to work on, and what needs improvement. The CMM tells you exactly where you are, and provides a bit of a roadmap, showing you how to get from your current situation to wherever you want to go.
Unsurprisingly, very few organizations achieve levels 4 or 5. Industry data suggests that nearly 90% of firms are levels 1, 2, or 3. I’d wager that the majority of dev-shops and startups are actually level 1. Ad-hoc practices, chaos, individual heroics. Either due to resistance to process change, lack of leadership support, or the costs associated with standardization. If you do graduate to the next levels the benefits are meaningful, but it’s a lot of work to get there.
The CMMI, or Capability Maturity Model Integration, emerged in the early 2000s as a response to some of the limitations of its predecessor. Unlike the original CMM, which was primarily designed for software development, CMMI was conceived as a broader, more flexible model that could apply across multiple domains, including software, services, and hardware development. CMMI represents an evolution rather than a complete departure from CMM. It retains the fundamental principles of process maturity but expands them to be more flexible, integrated, and aligned with the needs of a larger business. It introduces capability areas, continuous improvement, and a more adaptable structure, so many people see it as a more effective framework for most organizations. I personally think the original CMM is better (it’s simpler and more elegant), and that’s the one I use, but that's just me. Many orgs and departments have begun embracing CMMI instead. Including, recently DevOps orgs.
The authors of today’s study set out to examine a fundamental question about the relationship between CMMI and DevOps: Can DevOps be effectively assessed using CMMI, and if so, does this lead to meaningful process improvements? This is up for debate because there’s a bit of a tension there. A tension between CMMI and DevOps.
In this paper the authors sought to determine whether these two paradigms (one built for structured, controlled process refinement and the other designed for agility and adaptability) could be reconciled in a way that provided practical value to engineering teams.
To explore this, they conducted a systematic mapping review, an approach used to identify patterns, gaps, and trends across a large body of academic literature. Unlike a traditional systematic literature review, which seeks to answer a well-defined research question, a systematic mapping review aims to categorize and quantify existing studies, providing a broad view of the research landscape rather than deep analytical conclusions about specific causal relationships. The authors examined papers spanning more than a decade, selecting only those that explicitly addressed the intersection of DevOps and CMMI.
What they found was revealing: despite the existence of numerous DevOps maturity models inspired by CMMI, there was no single, widely accepted framework that explicitly integrated CMMI principles into DevOps workflows. Instead, the literature tended to use CMMI as an assessment tool rather than as a blueprint for implementing DevOps practices. Many organizations employed CMMI-derived maturity models to evaluate the sophistication of their DevOps processes, assigning levels to teams based on automation, feedback loops, and cross-functional collaboration. However, these maturity assessments often existed in isolation from CMMI’s core methodology. That is, while organizations were measuring DevOps capabilities through the lens of CMMI, they were not necessarily adopting CMMI-based improvements to guide their DevOps evolution. This disconnect suggests that while CMMI provides a structured way to evaluate maturity, it may not always be well-suited as a prescriptive framework for DevOps adoption.
Another important theme that emerged was the difficulty of balancing agility with structured process control. As I said earlier, DevOps, by its nature, thrives on adaptability. It encourages frequent experimentation, rapid iteration, and decentralized decision-making. In contrast, CMMI is built around well-defined process areas, distinct domains of organizational activity that are systematically refined to achieve higher levels of maturity. When many companies attempted to merge these two approaches they struggled to find a middle ground. They wanted the benefits of DevOps, particularly its ability to deliver software rapidly and with high automation, but they also needed the discipline and governance that CMMI enforces. This was particularly true in industries where either regulatory compliance or long-term maintainability is critical. The researchers noted that in some cases, companies retrofitted CMMI concepts to DevOps by emphasizing process measurement and risk management in their continuous integration pipelines. In other cases, firms selectively adopted aspects of CMMI, such as process audits or formalized improvement cycles, while ignoring its more rigid structural elements. This really just reinforced the idea that CMMI was being used as an evaluation tool rather than as an integral part of DevOps practice.
Interestingly some of the organizations using both DevOps and CMMI tended to develop their own internal hybrid frameworks rather than adopting a standardized integration model. These custom frameworks often borrowed from CMMI’s structure while maintaining the iterative flexibility of DevOps. Some teams built dashboards that mapped DevOps performance metrics to CMMI’s process areas. These allowed leadership to gauge whether their DevOps initiatives were progressing toward higher levels of organizational maturity. Others introduced governance mechanisms that ensured DevOps transformations aligned with broader business goals. That being said, no single approach dominated, and the lack of a standardized methodology for combining these two paradigms suggests that companies are still navigating this space through trial and error rather than following an established best practice.
Ultimately, this research highlights an important gap in the industry’s understanding of DevOps process improvement. While CMMI offers a mature and respected framework, it does not seamlessly align with the principles of DevOps. While many organizations are interested in using CMMI to measure their DevOps effectiveness, there is no consensus on how to fully integrate the two in a way that maintains both agility and structured process improvement. This raises deeper questions about whether new, hybrid maturity models are needed. Ones that preserve the rigor of CMMI while allowing for the dynamism that DevOps demands. Whether the industry moves toward such a hybrid approach remains to be seen, but for now, the integration of DevOps and CMMI remains more of an assessment practice than a true methodological fusion.
If you’d like to dive deeper into the article, explore their analytical framework or get a hold of the studies they based their analysis on, then I’d highly recommend that you download the paper.