Which Configuration Option Should I Change?

Author(s): Sai Zhang, Michael D. Ernst
Venue: Proceedings of the 36th International Conference on Software Engineering
Date: 2014

Type of Experiement: Controlled Experiment
Sample Size: 8
Data Collection Method: Observation


Modern software systems often offer users configuration options in order to allow them to modify behavior. As these software systems evolve with newer versions, previous configuration options may work differently and cause unexpected behavior for the user. The authors have created a tool called ConfSuggester in order to help users find and fix these configuration errors.

Zhang and Ernst ran ConfSuggester for 6 different software systems written in Java. There were 8 configuration errors that they had previously identified in the software before running their program. In 6 cases ConfSuggester’s top suggestion for the configuration error that caused the unwanted behavior was the correct choice. In 1 case it was ConfSuggester’s 3rd suggestion and for another it was its 6th suggestion.

ConfSuggester seems to be easy to use since it only requires instrumenting the Java files for the old and new version of the system and then demonstrating the unexpected behavior so the program can check for differences. ConfSuggester will output a list of suggested configuration options that could be causing the issue.

Configuration option changes are shown in the study to be very common in large systems. Users and system administrators may have to spend a lot of time trying to figure out their problem, but using this tool can minimize the amount of time spent fixing configuration issues. As developers this is good because the users can figure out issues without needing to know what is happening in the code.