Impact of Using Test-Driven Development: A Case Study

Author(s): S. Yenduri and L. Perkins
Venue: Software Engineering Research and Practice
Date: 2006

Type of Experiement: Case Study


The main objective of this experiment was to compare and evaluate the impacts between Test-Driven Development and Traditional Development, in regards to quality, productivity, and defects in a software program created by students.

A study was conducted at the University of Southern Mississippi by two groups of senior-level undergraduate students. There were nine members in each group. One group developed the piece of software using traditional develop first and test last approach, and the other group used test-driven development to complete the program. The program was done in Java and completed after three months. Both groups of students were trained in each approach, which brings up the question of if there was a proper control group in this experimental setting.

The results were analyzed by their performance metrics. The number of test cases written, the number of faults (defects) found, and the total number of hours spent. These metrics were used to evaluate quality, faults, and productivity, respectively. They found that the overall time taken by the TDD approach was 317 hours less than the traditional approach (1245 hrs vs 928 hrs). The TDD approach wrote roughly three times the number of test cases as the traditional approach (619 vs 211). The number of defects was measured in three categories: during unit tests, during integration testing, and during acceptance testing. The number of faults during unit testing was 74 in TDD vs 109, during integration testing it was 13 for TDD and 15 in Traditional, and in acceptance testing, TDD had 14 defects compared to the Traditional group of 31 defects.

There is no statistical analysis of these values to see if any of the differences are statistically significant, although the reports do show that the values are lower in all categories. The authors conclude that TDD increases quality, productivity, and lowers defects for small-sized software projects, but the usefulness in TDD for large-scale projects still must be analyzed. The authors also state that TDD is “clearly better” than traditional approaches, but again, I didn’t see any statistical analysis to prove that they can confidently state that it is clearly better.