Empirically-Guided Software Effort Guesstimation

Author(s): Philip M. Johnson, Carleton A. Moore, Joseph A. Dane, Robert S. Brewer
Venue: IEEE Software
Date: 2000

Type of Experiement: Controlled Experiment
Sample Size: 16
Class/Experience Level: Graduate Student
Participant Selection: class participation


Link: http://portal.acm.org/citation.cfm?id=626202

This study investigates the accuracy of several software development estimation techniques. The LEAP toolkit is used first to collect PSPish data; that is data points like effort expended (time) and lines of code generated. The toolkit uses these historical data points to generate estimates for future projects through several different calculation methods. The toolkit is flexible enough that the developer can “over-ride” a calculated estimation component with a guesstimate.

A major finding of the report was that the PSP’s PROBE method of estimation was far behind the leading estimation method of simply making an educated guess. PROBE was in 6th place. This is additional data that may discourage people from adopting the labor intensive PSP process.

The study also found that, far-and-away, developers liked to use their own estimations, rather than those calculated by a formal method. However, developers like to see the estimates calculated by several different methods to assist in coming up with their own estimate.

The authors state that their study’s sample size is too small to be able to make a strong claim regarding their results, and that replicated studies should be performed to see if correlating results are generated.