Continuous Deployment at Facebook and OANDA

Author(s): Tony Savor, Mitchell Douglas, Michael Gentili, Laurie Williams, Kent Beck, Michael Stumm
Venue: International Conference on Software Engineering
Date: 2016

Type of Experiement: Case Study
Class/Experience Level: Professional
Participant Selection: Analyzed and compared deployment data from two companies
Data Collection Method: Observation, Project Artifact(s)

Quality
5

This study analyzes and compares deployment data from two companies: Facebook and OANDA. Facebook is a large social media company with thousands of engineers and OANDA builds a currency trading system with roughly 100 engineers. This papers aims to show that continuous deployment does not inhibit productivity or quality as the size of engineering teams grow, management support for continuous deployment is vital to it's success and developer prefer faster deployment of the code they develop.

The paper begins with a general overview of Continuous Deployment and some techniques associated with Continuous Deployment. The general steps include: Testing, Code Review, Release Engineering and Deployment. Some techniques covered are: blue-green deployments, dark launches, staging/baking and shadow testing.

This case study presents data from Facebook between 2008 - 2014. At Facebook, they collected information on their Git repositories. For OANDA, they also collected Git repository information as well as information from their own deployment management system. The study attempts to measure productivity with lines of code as their metric. At Facebook, productivity was relatively constant from 2008 - 2014 while the number of developers increased by a factor of over 20 during that period. While there can be many factors can contributed to productivity, the authors concluded that continuous deployment does not prevent an engineering organization from scaling productivity as the organization grows and the product becomes more complex. The study also analyzed quality as a metric by tracking the number of critical, medium priority and low priority defects reported. The study observed that the number critical defects were relatively constant as the number of deployments per month increased while the number of medium priority defects have a slightly higher slope and the number of low priority defects scaled linearly with the number of deployments per month. With the switching of management teams at OANDA, the study observed a significant decrease (23%) in number of deployments when the management team did not support Continuous Deployment.

As a conclusion, the authors recognized that Continuous Deployment is a big effort and takes commitment and resources from the engineering organizations. Facebook commits about 5% of its resources to DevOps and OANDA commits about 15%. The study also emphasize the importance of giving developers freedom and decentralized decision making.

0