Improving Refactoring Speed by 10X

Author(s): Jongwook Kim, Don Batory, Danny Dig, Maider Azanza
Venue: International Conference on Software Engineering
Date: 2016

Type of Experiement: Controlled Experiment
Sample Size: 54
Class/Experience Level: Undergraduate Student, Graduate Student
Participant Selection: classwork for undergrad and grads
Data Collection Method: Code Metric


Modern Integrated Development Environments (IDEs) provide developers with automated refactoring tools that has changed how software is developed. However, scripting is a major feature that has yet to be implemented. Many Gang-of-Four design patterns can act as "refactoring scripts" because their nature is a series of programatic refactoring steps. When applying an GOF design it can be tedious and error prone if done by hand. The paper presents a design for a Java refactoring engine that can support refactoring scripts called R3.

The effectiveness and correctness of R3 was examined through two controlled experiments. 44 students in an undergraduate computer science course were asked to participate in the Adapter and Visitor experiments. The dependent variables were correctness of the application of the design pattern and time it took to complete. The independent variable was method used to refactor (R3 or regular refactoring tools provided by IDEs). The Visitor experiment was repeated with 10 graduate students.

The results were similar amongst both experiments. R3 was statistically more successful when applying the design patterns. It did take longer to use the R3 tools because the students had to write their own scripts rather than immediately using traditional refactoring tools. However, once a script is written it can be reused and students said writing the scripts helped them further understand the design patterns and strongly agreed that scripting would be a useful addition to an IDE.