Tricorder: building a program analysis ecosystem

Author(s): C. Sadowski, J. van Gogh, C. Jaspan, E. Soderberg, C. Winter
Venue: International Conference on Software Engineering
Date: 2015

Type of Experiement: Case Study
Class/Experience Level: Professional
Participant Selection: Google Engineers
Data Collection Method: Observation, Code Metric, Project Artifact(s)


Researchers at Google set out to develop a better static analysis tool with the following attributes: one that would actually be used by developers, integrate smoothly with existing workflows and scale to a system the size of Google. They found that current static analyzers were too inefficient or distracting to developers to have a place at Google. Developers would often ignore warning/error messages signaled by these tools if it required the developer to look away from their code. High false-positives, warnings about code that may be entirely superficial, also ruined developers workflow. A small team of 2-3 developers built TRICORDER, scalable, data-driven, program analysis platform. The key features of TRICORDER are it's non-impactful error messages, modularity, and scalability.

To measure the effectiveness of TRICORDER the researchers considered the four following attributes: Usability, Codebase Impact, Pluggability, and Scalability. Developers were typically happier with the less invasive analysis platform. They also noted that TRICORDER produced significantly fewer false-positives, implying that developers were able to implement positive changes to their code. What followed was an overall decrease in instances of violations in the Google codebase. There was also a notable increase in violations being prevented from entering repositories. TRICORDER was also found to scale very well to all of Google; TRICORDER is run every day against every snapshot produced by Google.

The implementation of TRICORDER offered Google an interesting methodology for their case study. Since no surveys or control groups were used, they were able to directly observe its impact within a practical setting, further solidifying their confidence in its success.