This paper described a study done with data gathered from two educational institutions for two different projects. In each case FIT was introduced as a required component of the project to be completed. Developers were given initial FIT test documents, were asked to ensure all tests passed, and asked to extend the test suite with new tests. Developers were formed into teams of 4-6 to complete the project. The following five hypotheses were tested:
Several methods were used to gather data from the projects. A modified FitNesse binary was distributed to the developers which recorded information from each test-run, including the timestamp and number of failed and passed tests. Additional information was gathered by inspecting both the source and test fixture code. A subjective rating was assigned to each of the fixtures, looking at the “fatness” of that fixture, in addition to categorizing the different fixture types. After doing this, the authors were able to determine the more commonly used fixture types as well as which types produced the least favorable “fatness” fixtures.
In analyzing the results, the authors focused on the differences between the two courses. The courses differed in when FIT was introduced, in the beginning of one course and in the middle of another. The primary result of this was that started with FIT produced fixture code with a higher “fatness”, likely because all of the fixture code was written before any of the implementation code was written. The students also favored the simplest type of fixture code, resulting from the high learning curve of the FIT framework.