Information needs for software development analytics

Author(s): Raymond P. L. Buse, Thomas Zimmermann
Venue: International Conference on Software Engineering
Date: 2012

Type of Experiement: Survey/Multi-Case Study
Sample Size: 110
Class/Experience Level: Professional
Participant Selection: Random selection from pools of email responders
Data Collection Method: Survey


Buse and Zimmermann surveyed 53 engineers and 57 lead engineers working at Microsoft in a "diverse set of project domains." The intent of the survey is to develop guidelines for creating software development analytic software for both software developers and software project managers in the future.

From the results of their survey, Buse and Zimmermann find that managers prefer to rely on data and metrics while developers prefer relying on personal experience when making decisions. Both developers and managers thought that predicting what will happen and the best and worst case scenarios is both more difficult and less important than knowing what happened and why and how it happened. Regarding the most important artifacts, managers valued features, bug reports, test coverage and code churn while developers payed more attention to code complexity and dependencies.

Buse and Zimmermann conclude that one tool for both developers and managers may not be adequate and even one tool for the duration of a project may not be adequate. Buse and Zimmermann recommend developing tools (or views within tools) specifically for managers, developers, and the current stage of the project (requirements gathering, construction, or maintenance). Above all, however, Buse and Zimmermann intend for this paper to be a call for more research regarding software development data and analytics.