An Empirical Study of the Evolution of an Agile-developed Software System

Author(s): A. Capiluppi, J. Fernandez-Ramil, J. Higman, H. C. Sharp, N. Smith
Venue: International Conference on Software Engineering. Proceedings of the 29th International Conference on Software Engineering 2007
Date: 2007

Type of Experiement: Case Study
Sample Size: 15


An Empirical Study of the Evolution of an Agile-developed Software System presents – according to the authors’ knowledge – the first measurement-based study of the evolution of software developed using an agile approach in an industrial setting. The organization studied and analyzed is a company which employed a medium-sized team to develop web software using extreme programming methodologies. The Java-based software and tools written by the company analyzes the content of a current web page that a user is navigating to determine the user’s interests in order to display related advertisements. The company was founded in May 1999, and this study was performed between October 2002 and March 2005, lasting two and a half years.

The paper uses the combination of quantitative and qualitative evidence to support the claims it makes. The authors assert that the majority of developer effort occurs in maintenance and evolution, and analyzed a copy of the entire code repository (130,000 lines of original code) for the project in order to gather data. The code was analyzed and extracted using the authors’ own toolset, the accuracy of which is unknown. The authors even include a disclaimer that “Despite our best efforts, errors in the data gathering and measurement extraction are possible and we cannot guarantee that our tools are error-free.” Additionally, complexity measurements are noted to not be fully representative of the actual system.

Smooth growth (with small perturbations) was seen in the evolution of this agile system. Growth rate measured in lines of code was higher than growth rate measured in files or directories. The final product’s quality was enhanced using extreme programming methodologies; as the managing director reported, clients were impressed with the “agile approach” because of the responsiveness of the team to their needs. The team managed to produce better quality software for every customer, but the effects of extreme programming on productivity was not discussed. Refactoring in order to maintain code simplicity directly resulted in higher quality code, and having an on-site customer allowed programmers to build a system that was constantly evolving toward the customer’s idea of the final system, removing the possibilities of producing code that implemented a misunderstood (and therefore incorrect) requirement.

The product studied in this paper was also a success commercially in that it survived well in the market for 6 years, continuing to use extreme programming for the entire time. Overall, this case study provides evidence that extreme programming allows smooth evolution of a software system while avoiding the problems of increasing complexity or decreasing customer satisfaction.