On Fault Representativeness of Software Fault Injection

Author(s): R. Natella, D. Cotroneo, J. Duraes, H. Madeira
Venue: Transactions on Software Engineering
Date: 2013

Type of Experiement: Controlled Experiment
Sample Size: 3
Class/Experience Level: Professional
Participant Selection: Frequently used safety critical software (MySQL DBMS, PostgreSQL, RTEMS)
Data Collection Method: Observation, Survey


This article discusses the representativeness of injected faults in fault injection software tests, and proposes approaches that increase the meaningfulness of results gained through software injection testing.

Due to the fact that software systems can be extremely complicated, and due to budget and time constraints, software will eventually run under faulty conditions that were not taken into account during testing. As a result, software engineers put systems in place in order to detect these faults, and one of the main ways that this is accomplished is by using testing their software systems with fault injection methods.

Hardware based fault injection techniques are recognized as reliable, but the complexity of current hardware makes the use of hardware based fault injection techniques not feasible. Instead, software based approaches (Software Implemented Fault Injection) has become more prevalent. This study focuses on showing how significant non-representative faults can be, discusses how fault representativeness is impacted by fault locations, and indicates that representativeness of faults can be improved by using certain algorithms and software metrics.

The study found after 3.8 million experiments that their n fault injection technique, called G-SWFIT, resulted in 14 and 23 percent representative faults for the Database Management system tests, and 72 percent representative faults in the case of the real time operating system tested. The system used by the study uses algorithms that analyze the components of the system to be tested and determines suitable candidate locations for fault injection based on classification algorithms. The two algorithms used are a decision trees algorithms and an algorithm called k-means clustering. The first algorithm is trained by providing examples of components to be selected, and the second algorithm does not require training, but uses fan-in and fan-out metrics in order to determine their goals. The article concludes that using G-SWFIT with these algorithms can both reduce fault-load size while improving fault representativeness.