The Personal Software Process (PSP): An Empirical Study of the Impact of PSP on Individual Engineers

Author(s): Will Hayes, James W. Over
Venue: Carnegie Mellon University Technical Report
Date: December 1997


This study makes the hypothesis that following the PSP will lead to:
• better estimating
• better planning and tracking
• protection against overcommitment
• a personal commitment to quality
• the engineers’ involvement in continuous process improvement

The study was conducted by administering a total of 23 PSP training courses to 298 engineers. This resulted in over 300,000 lines of code over a time span of more than 15,000 hours, with a resulting approximation of 22,000 defects. The standard validity threat of the difference in abilities and performance between engineers is a standard worry, and certainly exists in this study. The course incorporates a total of 9 assignments, over which individuals performance is tracked. The study prefers to focus on individual changes rather than group averages. To analyze this, the authors utilized the statistical method known as repeated measures of analysis of variance (ANOVA).

The results find that the PSP provides for relatively significant improvement in the first 4 areas of their hypothesis, without changing the 5th area. Specifically they found:
• Effort estimates improved by a factor of 1.75 (median improvement).
• Size estimates improved by a factor of 2.5 (median improvement).
• The tendency to underestimate size and effort was reduced. The number of overestimates and underestimates were more evenly balanced.
• Product quality, defects found in the product at unit test, improved 2.5 times (median improvement).
• Process quality, the percentage of defects found before compile, increased by 50% (median improvement).
• Personal productivity, lines of code produced per hour, did not change significantly. However, the improvement in product quality resulting from the PSP is expected to improve productivity and cycle time as measured at the project level (i.e., when integration and system test phase effort are included in productivity and cycle time).