Organization structure matters

Nagappan recognized that although metrics such as code churn, code complexity, code dependencies, and other code-related factors have an impact on software quality, his team had yet to investigate the people factor. The Mythical Man Month is most famous for describing how communication overhead increases with the number of programmers on a project, but it also cites Conway’s Law, paraphrased as, “If there are N product groups, the result will be a software system that to a large degree contains N versions or N components.” In other words, the system will resemble the organization building the system.

The first challenge was to somehow describe the relationships between members of a development group. The team settled on using organizational structure, taking the entire tree structure of the Windows group as an example. They took into account reporting structure but also degrees of separation between engineers working on the same project, the level to which ownership of a code base rolled up, the number of groups contributing to the project, and other metrics developed for this study.

The Influence of Organizational Structure on Software Quality: An Empirical Case Study, by Nagappan, Brendan Murphy of Microsoft Research Cambridge, and Victor R. Basili of the University of Maryland, presents startling results: Organizational metrics, which are not related to the code, can predict software failure-proneness with a precision and recall of 85 percent. This is a significantly higher precision than traditional metrics such as churn, complexity, or coverage that have been used until now to predict failure-proneness. This was probably the most surprising outcome of all the studies.