A Leveled Examination of Test Driven Development Acceptance

Author(s): D. Janzen and H. Saiedian
Venue: Proc. 29th Int’l Conf. on Software Engineering (ICSE)
Date: 2007


These days, many computer science and software engineering professors are starting to incorporate Test-Driven Development (TDD) into their curriculum. Little studies have been done to find out when the appropriate time and method is to start teaching these concepts. This study tries to figure out what level of education is optimal to begin teaching TDD through reports on students acceptance based on programmer maturity.

Between Summer 2005 and Spring 2006, six experiments were conducted. Five of them were at the University of Kansas. The last experiment was done at a Fortune 500 company. About 100 students participated from the Programming 1 (CS1) class. 30 more students from Programming 2 (CS2) participated. 12-15 students and professionals from graduate level courses and the Fortune 500 company were the remainder of the test subjects. Since not all students took the same courses over the semesters, the test subjects were not consistent across the board, but the experiment conducted and results analyzed were the same every time. The study had two hypotheses that it was trying to determine. The first hypothesis studied whether programmers perceived test-first as a better programming approach whether or not they had any prior experience using it. The second hypothesis studied whether programmers who have attempted test-first prefer it over a test-last approach.

The results statistically corresponded to the students level of academia. 86% of beginner programmers (those in Programming 1, CS1, and those in Programming 2, CS2) chose test-last as a preferred development approach. 87% of mature programmers (those of graduate status or in industry) chose test-first as a preferred development approach. Many stated that although they thought that test-first was better for programming, they thought it was much more difficult.

After analyzing the results, the most apparent conclusion is that mature programmers are more likely to choose TDD over a similar test-last approach. Mature programmers almost unanimously like and choose test-first as a development process. Interestingly, developers (both beginner and mature) who have tried test-first are more likely to choose test-first.