The first half of this paper describes the process and background of ATDD and a specific framework for this practice, EasyAccept. The second half of this paper gets into the description of the actual experiment using this ATDD framework.
The authors begin by describing the course where ATDD has been introduced. This is an undergraduate software engineering course scheduled to be taken in a students fourth semester; students are intended to have already taken and passed introduction programming and data structures courses and this is their first analysis course in object-oriented programming. Students are formed in groups of two-to-three and assigned a different project, with projects not getting repeated from semester to semester.
Prior to the introduction of ATDD, students in this course were initially given a general outline of work to complete, user story descriptions, and a glossary of relevant terms. Students implemented the projects based solely on these user stories and grading was done subjectively based on analysis of their design, code, and tests created. After introducing EasyAccept, students were initially given all of the acceptance tests for the project to be completed. Students were then graded on how many of these acceptance tests were passed.
To compare the course styles, the projects from previous courses were tested with ATDD retroactively and compared to projects using ATDD from the start. The authors found that projects from a previous class passed an average of 71.43% of acceptance tests. Projects from students using ATDD passed an average of 97.92% and 98.62%, over two classes. The authors were able to demonstrate that ATDD improves the overall correctness of a project, but not necessarily that ATDD works well in a class setting.