Current practice in measuring usability: Challenges to usability studies and research

Author(s): Kasper Hornbæk
Venue: International Journal of Human-Computer Studies
Date: Received 23 February 2004; received in revised form 27 May 2005; accepted 14 June 2005 Available online 15 August 2005

Sample Size: 180


Summary: This study reviews the current practices in measuring usability by categorizing and discussing usability measures from 180 studies published in core HCI journals and proceedings. From the review, the author identifies several challenges to usability including to empirically compare subjective and objective measures of usability, develop measures of learning and retention, study long-term use and usability, to measure long term satisfaction beyond post-use questionnaires, to validate and standardize the host of subjective satisfaction questionnaires used, to study correlations between usability measures as a means for validation, and to use both micro and macro tasks and corresponding measures of usability.

The author categorizes usability studies into measures of effectiveness, measures of efficiency, and measures of satisfaction. Each category is subdivided into several sub groups such as measures of accuracy, input rate, and ease-of-use. The author categorizes the usability studies into these groups in large tables by measuring the number of studies that fell into these categories. The author offers comments and analysis on the results of the categorization. After the review, the author identifies the challenges to measuring usability. He uses the evidence gained in his review to identify measures of usability that are lacking. In addition, he identifies weaknesses in the usability studies reviewed. Through this process the author identifies the challenges to measuring usability stated above. The author also warns against the misuse of usability studies as empirical evidence for quality-in-use. The author finishes by recognizing his own weaknesses in measuring only research studies and not industry and the neglect of context in the use of usability studies.