Automatic Test Generation: A Use Case Driven Approach

Author(s): Clementine Nebut, Franck Fleurey, Yves Le Traon, and Jean-Marc Jezequel
Venue: IEEE TRANSACTIONS ON SOFTWARE ENGINEERING
Date: 2006

Type of Experiement: Survey/Multi-Case Study
Class/Experience Level: Undergraduate Student, Professional
Data Collection Method: Observation

Quality
3

This was an interesting paper describing a technique to automatically create tests from a use-case. The technique used to generate these tests is first described in detail, including the various algorithms used for each step of the test generation. These steps include:

  1. Generating test objectives from use-cases, using pre and post condition logic. In others, determining objectives through a requirement-by-contract approach.
  2. Generating test scenarios from test objectives, used directly as an executable test.

This technique is not fully automatic and does require user involvement. The first big task of the user is to actually add the contracts to the use-cases, and this was one of the more challenging parts found during the case-studies. The actual test objective creation was also challenging, as different techniques and thought went in to ensuring different coverage levels. After generating the test scenarios, in some cases users needed to go back and manually adjust or add to the generated tests.

To test out this technique, four different case-studies were created around existing applications. Three of these were academic projects that could be closely controlled; the fourth an industry project. In all of the case-studies, the actually study was divided into three parts:

  • Requirements stage – Define and attach a scenario for each use-case and verify generated contracts.
  • Test generation stage – Based on the defined criteria, test objectives are created.
  • Test execution stage – Run the tests and record the results

This paper then compared and evaluated each of the studies in terms of overall test coverage and efficiency in test coverage. While this technique showed promising results, a more efficient criteria for both test generation and test efficiency measures needs to be researched. In the industry example one of the important lessons learned was that only 79% of the existing requirements could be represented in use-cases. This point needs to be researched further to reveal any negative impact of using this technique in practice.

0