A Comparative Study of Two Usability Evaluation Methods Using a Web-Based E-Learning Application

Author(s): Samuel Ssemugabi, Ruth de Villiers
Venue: ACM International Conference Proceeding Series; Vol. 226
Date: 2007

Type of Experiement: Case Study


Pages: 132 - 142
DOI Bookmark: http://doi.acm.org/10.1145/1292491.1292507

The authors of this paper seeks to compare two methods of heuristic evaluation of usability between experts with expertise in the area of HCI and end users. The authors investigate the extent in which the heuristic evaluation identifies usability problems in a web-based learning application. The authors find that 4 expert evaluators identified more problems than 61 non-experts and identified 91% of the non-experts problems. The authors conclude that heuristic evaluation by competent and experts was an appropriate and efficient method of usability evaluation for e-learning applications.

For the experiment, an appropriate framework of evaluation criteria is used as the basis of an heuristic evaluation performed by experts and in a survey among end-users. Each of the expert evaluators conducted their evaluations separately. Only one of the four experts directly evaluated the website. The other three served as objective observers mostly to expedite the process of evaluation. There was a tendency of different evaluators to concentrate on different heuristics. Experts found 58 problems compared to 55 by the non-experts.