A practical guide to testing the understandability of notations

Author(s): Susanne Patig
Venue: Proceedings of the fifth on Asia-Pacific conference on conceptual modelling - Volume 79
Date: 2008


DOI: http://portal.acm.org/citation.cfm?id=1379429.1379438 (DOI link unavailable)
Type of Experiment: Empirical Study

The work presents a "guideline for planning and conducting" experiments to test the precision and understandability of model notations. The authors also include an open-source tool, /notate/, which can be used for experiments on understandability. As proof of the importance of this work, the authors give experimental results showing their guideline and tool applied to one of SAP AG's metamodels.

Study Details
The main experimental goal was to determine the differences in precision and understandability between a graphical notation and a textual one for the metamodel. The study follows a within-subject design with specifics listed below.

Hypothesis: There is no difference in the understandability of the graphical and the textual notation of the SAP metamodel for complex data types.
Sample Size: 15 participants
Sample: 42 subjects (35 male, 7 female); 6 had graduate degrees in computer science, 36 students (17 undergraduate, 19 graduate) in computer science, computational visualistics, data and knowledge engineering, and business informatics