: Raymond P. L. Buse, Thomas ZimmermannVenue
: Software Engineering (ICSE), 2012 34th International ConferenceDate
: 2012Type of Experiement
: Survey/Multi-Case StudySample Size
: 110Class/Experience Level
: ProfessionalParticipant Selection
: Random selection of participants were given the survey to respond to.Data Collection Method
Many aspects of Software Engineering can be measured automatically, but prediction is still hard to do, making Software Development risky, so many Software Projects are delayed or never finished. Analytics can help developers and managers to make informed decisions based on data to reduce risk. There has been research done on the information needs of developers, but not managers. The paper focuses on the information needs of managers to achieve analytics that will help managers make project decisions.
The authors propose a study on the information needs of 110 developers and managers at Microsoft. They authors hypothesis that analytics can help answer important questions about software projects. They conducted a 28 question survey via email to determine the information needs for project decisions. The survey focused on identifying which factors were most important to decision making, having the participants rate them from not important to very important on a 4 point scale. They also rated the importance of different artifacts (test case, feature, etc.) on a 4 point scale. They were also asked which indicators they would use if they were made available, "failure information was ranked most important and bug reports second most overall".
The survey also had participants describe the scenarios in which analytics were useful and responses were summarized into main categories: targeting testing, targeting refactoring, release planning, understanding customers, judging stability, targeted training and targeting inspection. These categories are listed in order of most frequently mentioned.
Based on their results, the authors present a set of guidelines that any analytics tool should meet and identify analysis types that fit with their identified information needs. Analysis types are: trends, alerts, forecasting, summarization, overlays, goals, marketing, benchmarking and simulation. After their survey the authors explain that they believe analytics will help managers make informed software decisions and urge the community to develop new analytics tools.