Managing Technical Debt in Enterprise Software Packages.

Author(s): Narayan Ramasubbu, Chris F. Kemerer
Venue: IEEE Transactions on Software Engineering (Volume:40 , Issue: 8)
Date: 02 June 2014

Type of Experiement: Survey/Multi-Case Study
Sample Size: 69
Class/Experience Level: Professional
Participant Selection: Participants were selected from a series of Fortune-500 Software Companies based on availability and willingness to participate.
Data Collection Method: Observation, Survey

Quality
4

In Managing Technical Debt in Enterprise Software Packages, Ramasubbu and Kemerer explore links between the amount of technichal debt found in a code project, and the rate that the team jumps on new technologies. In this paper, Ramasubbu and Kemerer outline several ways to characterize groups of software engineers as Base, Early, or Late adopters. Base adopters tend to use an Off-The-Shelf piece of software as intended, not adding functionality, and attempting to make the most of the API as possible. Early adopters write custom code to add functionality, often staying ahead of the software for a few years. Late adopters tend to lag behind the software during it's lifetime, but add functionality once a package stops focusing on adding features.

In this paper, Ramasubbu and Kemerer attempt to prove the following 6 hypothesis regarding the groups:

  • "1A) Early adopters will be less likely to add features late in the package’s lifespan, and the final cumulative amount
    of functionality in package variants of early adopters will be lower than that of base and late adopters"
  • 1B) "Late adopters will be more likely to add more features late in the package’s lifespan, and the final cumulative amount of functionality in package variants of late adopters will be higher than that of base and early adopters."
  • 2A) All else being equal, throughout the evolution of the software package software quality of early adopter package variants will be lower than that of the base package."
  • 2B) All else being equal, throughout the evolution of the software package the software quality of late adopter package variants will be higher than that of the base package.
  • 3A) All else being equal, customer satisfaction of the early adopter package variants will be higher than that of base adopters before the point of takeoff and lower than base adopters after the point of saturation.
  • 3B) All else being equal, customer satisfaction of the
    late adopter package variants will be lower than that of base
    adopters before the point of takeoff and higher than base adopters
    after the point of saturation.

For reference, the following terminology is used in the paper:
A Package is an off-the-shelf piece of software that accomplishes some business goal. Packages undergo two major stages of development: Take off and Saturation Take off occurs when a significant portion of users begin using the software, and the package developer commits to a long-term feature plan.. Saturation occurs when major features of a package have been completed, and the package maintainer can no longer continue to add them.

A Transaction is a new feature added to an existing software product by use of an API or other interface. Transactions are created by companies to tailor packages to their needs.

Through a study of 69 companies via survey and observation, this paper comes to the following conclusions:

  • Ramasubbu and Kremerer found that a negative correlation existed between the number of transactions implemented during the infancy of a new package were inversely related to the number of transactions implemented past the point of saturation. Late adopters ended up developing features well after a package reached the point of saturation. This finding gives strong evidence to hypothesis (1a) and (1b).
  • Early adopters tended to have more bugs than the base program near the end of a package's lifetime. Late adopters tended to have less bugs than the base package. Both of these works give strong evidence in support of hypothesis (2a) and (2b)
  • Customer satisfaction tended to be higher when transactions were added during the early stages of a package's development. However, early adopters had trouble keeping customers satisfied after the saturation point of a package. Early adopters often found themselves with less transactions than the base package due to the inability to upgrade or modify code. This factor leads to lower satisfaction. Late adopters had issues keeping customers satisfied before packages reached saturation, but had an overall positive customer response after the point of saturation. These findings give strong evidence in support of hypothesis (3a) and (3b).

Overall, this paper offers strong evidence that becoming an early adopter can lead to higher customer satisfaction initially, but code debt can harm a team later in the packages lifespan due to unsupported code, inconsistencies and assumptions. Becoming a late adopter means lower customer satisfaction at first, but the cost is offset by the ability to develop faster, more reliable code late into a package's lifetime.

Ramasubbu and Kremerer conclude by offering managerial advice to manage code debt stemming from extending a package, and recommend a systematic comparison against the base package in three key timeframes: (1) the first instance when a package vendor announces a package roadmap, (2) whenever the cost of quality exceeds the cost of new functionality implementation, and (3) the first instance when functionality growth slows below an established threshold. Doing so can help minimize code debt from building up and preventing future growth.

0