Experimental Comparison of the Comprehensibility of a UML-based Formal Specification versus a Textual One

Author(s): Rozilawati Razali, Colin F Snook, Michael R Poppleton, Paul W Garratt, Robert J Walters
Venue: Proceedings of 11 th International Conference on Evaluation and Assessment in Software Engineering
Date: 2007

Sample Size: 41
Class/Experience Level: Graduate Student
Data Collection Method: Survey


This paper studied the comprehensibility of a UML based representation of software specifications vs. a textual representation of software specifications. The metric used in the experiment is the efficiency of preforming the comprehensibility tasks. The results of the experiment showed that semi-formal notation such as UML expedites the comprehension of software representations.

Experiment Design
41 3rd year computer science master students were divided up into 2 groups. Four models that represented two case studies were used. In the 1st case study the 1st group was given a UML model while the 2nd group was given a textual based model. In the 2nd case study the 1st group was given the textual model while the 2nd group was given the UML model. Participant were given 100 minutes to completed a questionnaire based on the case studies.

“The results indicate with 95% confidence that a UML model group could be up to 16% (overall comprehension) and 50% (comprehension for modification task) easier to understand than the corresponding textual based model group.” There was also evidence to support that one can still comprehend the notation in a UML-B model even with very
limited hours of training.