Automatically Recommending Peer Reviewers in Modern Code Review

Author(s): Motahareh Zanjani, Huzefa Kagdi, Christian Bird
Venue: Transactions on Software Engineering
Date: June 2016

Type of Experiement: Case Study
Sample Size: 4
Class/Experience Level: Professional
Participant Selection: They looked at code reviews of 3 open source and 1 closed source project
Data Collection Method: Observation

Quality
4

The point of this study was to focus on how to find peer reviewers who were most likely to contribute valuable information in a review. They were able to conclude that developers who were familiar with the code being reviewed completed reviews faster and more thoroughly. The approach to solving this problem was cHRev which focuses on code review histories to recommend future reviewers. Comparing this method to several other methods, the results showed that cHRev’s method of factoring in additional review contributions (such as feedback comments) and the recency of their involvement were much more satisfactory than the other methods.

The systems that they chose to use as data were 3 open source (Android Platform, Eclipse, Mylyn) and 1 closed source (MS Office) project. Taking a list of resolved changes and code reviews, they went back and ran each algorithm for generating a list of potential reviewers and from there calculate the accuracy for each method based on the number of developers on the list who actually completed a review. The results showed that cHRev showed statistically significant advantages with regards to accuracy and consistency. Some reasons that they believe that this method is more successful is because developers who have never touched the code directly before could still have significant knowledge in the area. One example of this being in the Android Platform project where very few people have permissions to alter the code, but more people have the ability to create patches to introduce new functionality.

0