On the Need for a Process for Making Reliable Quality Comparisons with Industrial Data

Author(s): Laurie Williams
Venue: ACM SIGSOFT Software Engineering Notes
Date: 2004

Type of Experiement: Case Study


Methodology: TDD
Duration: 1 year
Company: IBM
Type of project: Device Drivers
# of developers: 9
project life: New
Experience with the Methodology: None
Team Language Experience: 3 unexperienced
Language: Java and C
Team Domain Experience: 2 Novice to the device domain, 7 experienced
Collocation: Distributed
Technical Leadership: No Dedicated coach
% lower defect density: 40
Test Cases Run: .49 E
Test Cases/ Total LOC: .48 TC
Defects/Test Case: .36 TCL
Defects/KLOC: DFL

Methodology: XP
Duration: 1 year
Company: Sabre Airlines
# of developers: 10
Internally-Visible Quality (test defects/KLOEC of code): %35 of before
Externally-Visible Quality (test defects/KLOEC of code): 70% of before

This was a study that compared two different practices, XP and TDD. The goal of this was to suggest a good composite measure to compare between different process and study. The suggested measure is the Putnam productivity parameter (PPP) where PPP = (SLOC)/[(Effort/B)*(1/3) * (Time)*(4/3)]. Effort is the staff years of work done on the project. B is a factor that is a function of the system size, chosen from a table constructed by Putnam based on industrial data. SLOC is the source lines of code, and Time is number of elapsed years of the project.