A Controlled Experiment Measuring the Effects of Personal Software Process (PSP) Training

Author(s): Lutz Prechelt, Barbara Unger
Venue: IEEE Transactions on Software Engineering
Date: September, 1999


This experiment makes a comparison of the performance of participants who had just previously completed a PSP course to a control group of participants who had received other technical training instead. The experiment finds that while there may be some improvements, the results are much smaller than PSP proponents usually assume. Specifically, they found no improvement in effort estimation capabilities and only small improvement in program reliability.

The experiment was run on a total of 48 people, of which 29 were in the PSP group and 19 were in the control group. Each of these participants was a master student in Computer Science. The PSP group hand previously participated in a 15 week graduate lab course introducing PSP methodology, and the control group had previously participated in previous lab courses creating components in Java, but not using PSP. The experiment was conducted 0-3 months after the students had completed their respective previous labs. The estimated median of experience for each of the students was 8 years. Of all of the participants, 8 did not complete the experiment and their results were ignored.

The experiment lists several threats to validity, but generally concludes that PSP training is not as effective as its proponents believe it to be, likely because of the lack of adoption once that training has ended.