Classifying and Qualifying GUI Defects

Author(s): Val´eria Lelli, Arnaud Blouin, Benoit Baudry
Venue: 2015 IEEE 8th International Conference on Software Testing, Verification and Validation (ICST)
Date: April 13-17, 2015

Type of Experiement: Case Study
Data Collection Method: Observation, Code Metric


User interactions on software systems require a robust Graphical User Interface (GUI). But with these interactions with the GUI’s come bugs and other issues that cause the application to behave incorrectly. Extensive testing methods have been implemented to check for bugs to avoid these GUI failures. But new types of interactions in GUI’s get implemented which make software system prone to more and different errors. The article proposes that software testers need to create a comprehensive intensive vault of GUI faults. It goes into detail about 1) Designing and classifying GUI faults 2) Empirical analysis to see how important the proposed vault of GUI faults vs the failures found in real GUI’s. 3) Empirical analysis of GUI testing tools against failures from software systems and 4) Mutants have been developed based on the vaults of GUI faults (They are available freely to be reused by developers).

The article concludes with providing a GUI fault model for software testers so that they can check how reliable the the testing tools of the testers are. The fault model has been tested and proven with test data on 279 GUI bug reports from different open-source GUI’s. The article says that it proves this by creating mutants, or incorrect UI behavior, for Java open source GUI. Two different GUI testing tools were then used to see if they would catch those UI bugs. These errors occur in the first place because a) the GUI rendering is difficult to test since its based on code analysis and its very difficult to test graphical properties and 2) GUI’s are being composed of more complex interactions rather than the standard widgets which make it difficult to test.