Supporting the Evolution of a Software Visualization Tool Through Usability Studies

Author(s): Andrian Marcus, Denise Comorski, Andrey Sergeyev
Venue: Proceedings. 13th International Workshop on Program Comprehension, 2005. IWPC 2005.
Date: 15-16 May 2005

Type of Experiement: Survey/Multi-Case Study
Sample Size: 36
Class/Experience Level: Undergraduate Student, Graduate Student


Pages: 307 - 316

This paper presents a usability study on the software visualization tool sourceviewer3D (sv3D). Sv32 is designed to aid in understanding large software projects by displaying source code and associated metrics along three dimensions. The authors state, "The goal of the study is two-fold: to assess the affects of using sv3D as a new technology to support program comprehension and to gather feedback information from users, which will support further development of sv3D."

The usability study was conducted with 36 graduate and undergraduate computer science students. Twelve were used in the first pilot phase and 24 in the actual study. The authors identified know issues dealing with the transition phase of adopting a visualization tool that would likely affect their result. The participants were divided randomly into two groups: one which would answer questions using sv3D, and another that would answer using tabulated data. The students were asked to answer questions about a sample project such as "How many classes does this project contain?" and "Which is the largest class (in terms of lines of text)?" In addition, the participants were asked a number of questions to evaluate the features of sv3D.

The authors analyzed how many questions were answered correctly and how long it took the participants to answer the questions. The results show that the users from the sv3D group had slightly higher accuracy than the text only group, but not enough to be statistically significant. The authors were surprised to find that the sv3D group took significantly longer to answer the questions than the text only group. The authors theorized that the sv3D group took longer mostly due to their lack of familiarity with the tool.