Any empirical evidence on the efficacy of CMMI?

Question

I am wondering if there are any studies that examine the efficacy of software projects in CMMI-oriented organizations. For example, are CMMI organizations more likely to finish projects on time and/or on budget than non-CMMI organizations?

CMMI stands for "Capability Maturity Model Integration". It's developed by the Software Engineering Institute at Carnegie-Mellon University (SEI-CMU).

It's not a certification, but there are various companies that will "appraise" your organization to various levels of CMMI, such as level 2 and level 3. (I believe CMMI level 1 is an animalistic, Hobbesian free-for-all that nobody aspires to. In other words, everybody is at least CMMI level 1, even if you've never heard of CMMI before.)

I'm definitely not an expert, but I believe that an organization can be appraised for CMMI levels within different scopes of work: i.e. service delivery, software development, foobaring, etc. My question is focused on the software development appraisal: is an organization that has been appraised to CMMI Level X for software projects more likely to finish a software project on time and on budget than another organization that has not been appraised to CMMI Level X?

However, in the absence of hard data about software-oriented CMMI, I'd be interested in the effect that CMMI appraisals have on other activities as well.

I originally asked the question because I've seen various studies conducted on software (e.g. the essays in The Mythical Man Month refer to numerous empirical studies, as does McConnell's Code Complete), so I know that there are organizations performing empirical studies of software development.

Answer 1

The SEI has published some material regarding the impacts of CMMI on various aspects of software development. There are a few samples of benefits that companies have seen from adopting either CMMI or the SW-CMM. In addition, some companies have provided their success stories to the SEI. Of particular interest might be the technical report Demonstrating the Impact and Benefits of CMMI: An Update and Preliminary Results. Although these are all curated by the SEI, I tend to trust this data since the SEI is a federally-funded research and development center with significant ties to the academic and government communities rather than a commercial/for-profit entity.

You might also be interested in the January/February 2012 issue of CrossTalk, which is a journal for defense industry software engineering. This issue focuses specifically on the benefits of achieving high CMMI maturity (typically defined as Level 4 and Level 5) and reveal data from a couple of defense contractors on the impact to project cost, schedule, and quality (all positive) as they moved into the high maturity levels.