A Formal Experiment Comparing Extreme Programming with Traditional Software Construction

Author(s): Macias, F., Holcombe, M., Gheorghe, M.
Venue: Proceedings of the Fourth Mexican International Conference on Computer Science
Date: 2003

Type of Experiement: Controlled Experiment
Sample Size: 93
Class/Experience Level: Undergraduate Student, Graduate Student


A Formal Experiment Comparing Extreme Programming with Traditional Software Construction presents an experiment conducted on computer a science student that was carried out during the Spring academic semester at the University of Sheffield. The aim of the experiment was to assess extreme programming and compare it with a traditional approach. Ten teams worked with extreme programming and ten with the traditional approach, building software for real-world industry clients. In terms of quality and size, teams working with extreme programming produced similar final products to traditional teams. In spite of the absence of design and the presence of testing before coding, the products obtained using extreme programming had similar quality and size to products produced while following a more traditional process.

The students were closely observed during the academic semester and their work was measured using a variety of comprehensive metrics. Ninety-three students were enrolled in the course, and received training randomly in one of two areas: extreme programming, or the traditional waterfall process. As an important statistical note, surveys revealed that no student had prior industry experience.

According to the time spent (information from time sheets) and the activities (information from minutes), extreme programming teams spent less time in programming and more time in testing, while traditional teams spent more time in programming and less time in testing. People working in the extreme programming approach spent much less time in Analysis and Design, but the total amount of time spent in the project was overall higher in the extreme programming teams. Traditional teams spent much more time in analysis and design than extreme programming teams and these extreme programming teams produced very small, simple designs, if any. And, although slightly more time-consuming than its counterpart, extreme programming remains a low technology requiring process, making it less expensive than other approaches that require expensive, elaborate, or more sophisticated technology.

The experiment used mathematical models to ensure, by analyzing reports computationally, that each of the two software engineering processes was not deviated from, and that results had valid statistical value. The final results showed that β€œThe use of extreme programming in a software construction process produces as good results (external and internal quality) as those obtained from the use of traditional processes (in the average case of a small/medium size project).” The implication for extreme programming, then, is the possibility of growth and maturation given the fact that it provided results that were as good as those from the traditional approach.” The authors asserted their surprise that, even though extreme programming is a far newer concept than the traditional software engineering methodologies, it provides as good a result as traditional approaches, and that a procedure free of Design provides as good results as one including Design.