An empirical study of factors that affect user performance when using UML interaction diagrams

Author(s): J. Swan, T. Barker, C. Britton, M. Kutar
Venue: International Symposium on Empirical Software Engineering
Date: Nov. 2005


This paper attempted to determine whether sequence or collaboration diagrams were more
efficient to comprehend. Both are UML interaction diagram types that represent the
same types of data but do so differently. The authors also wanted to see how
user preference affects the efficiency of the diagrams. In addition, they wanted
to see how mixing horizontal and vertical text in the diagrams would affect the results.

40 people participated in the study. They were a mixture of final year CSC students,
CSC postgraduate students, and members of the university staff (Univ of Hertfordshire)
from either the CSC department or the psychology department. All of them had previous
experience with UML. They were shown a random set of UML diagrams that were either
sequence diagrams or collaboration diagrams. Each of the diagrams modelled aspects of
either using an elevator or an ATM. The participants answered a number of questions
on the diagrams and the system they modeled. They were timed for how quickly they answered
each question but were not told that they were being timed. The participants were
asked both before and after the experiment which diagram type that they preferred
(or if they had no preference).

The results were compiled from their responses and how quickly they gave them. Interestingly
enough, those with preferences for sequence diagrams (be it a pre or post-test preference)
did better with sequence diagrams. That would tend to imply that the participants knew
enough ahead of time to either know that they preferred sequence diagrams or to be able
to reasonably guess that they did. Those that did not prefer sequence diagrams after the
test performed equally well with either diagram type. However, those that did not
have a preference for either diagram type or preferred collaboration diagrams did
significantly better with either type than those that preferred sequence diagrams.
Maybe the smart people either don't care or just like collaboration diagrams better
for some reason. Whether the diagrams used horizontal text or a mixture of horizontal
text and vertical text had no effect on the results.

All in all, it looks to me that the sequence diagrams are probably easier to use if
you aren't very familiar with either diagram type, but if you know them both, then
you can be equally efficient with either. Certainly, the hypotheses that users
would do better with the diagrams that they prefer appears to be valid. Of course
that raises the question as to why they prefer that diagram type and how
those reasons affected the results.