Prioritizing Manual Test Cases in Traditional and Rapid Release Environments

Author(s): Hadi Hemmati, Zhihan Fang, Mika V. Ma ̈ntyla ̈
Venue: 2015 IEEE 8th International Conference on Software Testing, Verification and Validation (ICST)
Date: 2015

Type of Experiement: Case Study
Sample Size: 13
Class/Experience Level: Graduate Student
Participant Selection: Latest version of Mozilla Firefox
Data Collection Method: Code Metric, Project Artifact(s)


The authors of this study investigated the benefits of prioritizing hand written test cases against automated test cases. There are many different way to prioritize test cases (most of the time this is done by severity). It is useful to prioritize test cases so that you can identity where a program breaks at the earliest point rather than somewhere later down the chain of command.

The results were that if you are doing traditional software deployment (where development is in large increments) there is no significant difference in test case performance with prioritizing hand written test cases. However, if you are using Rapid Release (where develop is in small incremental stages), then prioritizing hand written test cases can speed up testing and allow you to develop faster. The author's came to this conclusion after prioritizing test cases on different Mozilla Firefox releases. They tested their hypothesis on 5 traditional deployments and 9 rapid releases of Firefox to find these results.