Documenting Design-Pattern Instances: A Family of Experiments on Source-Code Comprehensibility

Author(s): C. Gravino, M. Risi, G. Scanniello, and G Totora
Venue: International Conference on Software Engineering)
Date: 2015

Type of Experiement: Controlled Experiment
Sample Size: 15
Class/Experience Level: Graduate Student, Professional
Participant Selection: students currently enrolled in a particular course in partucular colleges, contact network of professionals in research group
Data Collection Method: Survey, Code Metric


This study compared the effects of source code comprehensibility with and without design pattern documentation. 4 experiments were conducted, with participants ranging from graduate level students, to PhD level, to professionals in industry. In these experiments, the independent group received the source code to comprehend along with either textual documentation, or visual documentation via UML diagrams. The control group received only source code. Both groups were permitted to use the aforementioned sources, prior knowledge, and the Internet in order to comprehend the codebase, but were conformed to use a specific text editor in order to read the source code.

Results were measured based on a score that took into account the correctness of understanding, the time taken, and the individuals' perceived confidence and difficulty. Participants (except the professional group) were prepped beforehand with a training session detailing the experiment and designed-pattern based development. comprehension questions were given to answer during the experiment, and participants were were given an exit survey to give their subjective impressions.

Overall, documenting design-pattern instances yields an improvement in correctness of understanding
source code, for both the textual documentation and (even more so) for visual documentation. Overall, all but the least experienced group showed sign of statistically significant improvement of the source code, with difference being directly correlated to experience level.