Using PSP to evaluate student effort in achieving learning outcomes in a software engineering assignment

Author(s): Brian R. von Konsky, Jim Ivins, Mike Robey
Venue: Australasian Computing Education Conference (ACE)
Date: 2005

Sample Size: 148
Class/Experience Level: Undergraduate Student
Participant Selection: class participation
Data Collection Method: Project Artifact(s)


Type of Experiment: Experience Report

This paper relates an experience using the PSP method to collect performance data. The collected data was then used for a separate study. The main focus of the paper is a study on effort expended by students to complete an assignment. The PSP was simply the mechanism used to collect the data. However, the authors did perform an analysis of the PSP’s efficacy as a tool for measuring programming effort.

Students involved were from the Engineering, Computer Science, and Information Technologies fields. The students used the PSP method to collect their time spent on a major assignment. The students recorded start, stop and interruption times, and lines of code created. The authors found that they weren’t highly confident in the quality of the data collected by the students using the PSP method. Through a simple analysis they determined that over 25% of the PSP logs contained data that was likely fabricated and inaccurate.

This is not a significant study of the PSP method. The analysis methods weren’t rigorous and the PSP data collection was really just a means to another end rather than a focal point of the study. Though not explicitly stated, the assumption is that the students weren’t all graduates of a PSP course.