An Empirical Study on the Relationship between Defective Requirements and Test Failures

Author(s): Ferguson, Robert W., and Giuseppe Lami
Venue: 30th Annual IEEE/NASA Software Engineering Workshop SEW-30 (SEW'06)
Date: 2006

Class/Experience Level: Professional


The paper is an empirical study to identify a relationship between expressiveness defects in natural language requirements and failures during testing. "Expressiveness" is the ability to convey the intended meaning by avoiding ambiguities and readability problems. The defect categories for expressiveness analysis are non-ambiguity, understandability, and specification completion. The paper concludes that “the test cases related to requirements containing expressiveness defects are more prone to produce failures.”

What type is the study? Industrial Software Project
What were the subjects? NASA Requirements Document
What software or prototypes did the study used? QuARS (Quality Analyzer for Requirements Specification) provides practitioners to identify “potential linguistic defects in natural language requirements and calculates a set of quality metrics automatically.”

Process Outline

  1. NASA requirements are identified and analyzed to find possible expressiveness defects with the QuARS tool
  2. NASA documentation package contains information tracing test failures to requirements to be assembled for analysis
  3. A comparative analysis between the requirements containing expressiveness defects and those associated with failure-causing modules is performed

The density of failure-causing requirements for the QuARS-defective requirements is 23.3% greater than the requirements not QuARTS-defective. The distribution of the failure-causing requirements varied for different software components. There is also a relationship between expressiveness defects and testing failures.