Using Software Testing to Move Students from Trial-and-Error to Reflection-in-Action

Author(s): S. Edwards
Venue: ACM SIGCSE Bulletin
Date: March 2004


According to Edwards, students have a fundamental problem of learning application and synthesis skills (writing code) without learning comprehension and analysis skills (testing). His vision is to move students software engineering approach of trial-and-error to reflection-in-action. Edwards believes that Test-Driven Development (TDD) taught in schools can achieve this.

A comparative study between students was conducted between quality of programs developed with Test-Driven Development methods, and those with traditional coding and encouraged testing. This experiment was done on a group of 118 students at Virginia Tech, sponsored by the National Science Foundation, and a research fellowship from Virginia Tech’s Institute for Distance and Distributed Education. 59 of the students were in a Spring 2001 Junior-level Computer Science course as the control group, and another 59 students were in the same course but taught with Test-Driven Development methods in Spring 2003. The course consists of 4 programming assignments, each taking 2-3 weeks to complete. In this study, only 1 of those programming assignments were assessed. Furthermore, of the total number of programs written, only 18 of them (9 from the control group, and 9 from the TDD group) were analyzed for total defect statistics. However, the results were promising.

Edwards found that the TDD students programs contained 45% fewer defects, had an average 5.9% higher of a grade on the assignments, and 17.2% higher score on a Web-CAT assessment which checks the correctness of code as well as quality of tests, code coverage, etc. His results were statistically significant, as they were analyzed with proper t values of 95% confidence.
Furthermore, from results of student surveys, he found that 65.3% of the students felt that using TDD increases the students confidence in the correctness of their code, and 67.3% felt that using TDD also increase confidence students have when making changes to their code.

This study shows that it is possible to teach TDD in a junior-level course and see promising results. Furthermore, the students agree with most of the TDD teachings. According to Edwards, TDD is a solution to developing comprehension and analysis skills as well as application and synthesis skills.