Would Static Analysis Tools Help Developers with Code Reviews?

Author(s): S. Panichella, V. Arnaoudova, M. Di Penta3, G. Antoniol
Venue: International Conference on Software Analysis, Evolution, and Reengineering
Date: 2015

Type of Experiement: Survey/Multi-Case Study
Sample Size: 6
Class/Experience Level: Professional
Participant Selection: Open source project contributors
Data Collection Method: Project Artifact(s)


These researchers were interested in how helpful static analysis tools would be to developers doing code reviews. Static Analysis tools analyze source code and provide different types of warnings to developers. These warnings can be related o design, code style, documentation, and much more. Many studies have looked at how these warnings affect software evolution but this study wanted to look at the tool’s affect on code reviews.

Without taking into account static analysis tools, code review compared to general software evolution removes a lightly higher percentage of warnings. The study mined six Java open source projects including Eclipse JDT Core, Eclipse Platform UI, and Motech. The first research question was “To what extent warnings detected by static analysis tools are removed during code reviews?” and the second question was “What kinds of warnings detected by static analysis tool are mainly considered during code reviews?”.

The results found that about 6% to 22% of warnings were removed during code reviews. The more interesting part of the results, though, is that specific types of warnings generated significantly higher removal percentages. Specifically, the imports, regular expressions, and type resolution categories of warnings all had removal percentages of over 50%.

The researchers conclude that enforcing the removal of some warnings found by static analysis tools could reduce effort and time needed for the code review process. They admit that static analysis tools may provide too many false positives if it is a broad warning detection tool, though.