Industry Application of Continuous Integration Modeling: A Multiple-Case Study

Author(s): Daniel Ståhl, Jan Bosch
Venue: International Conference on Software Engineering
Date: 2016

Type of Experiement: Survey/Multi-Case Study
Sample Size: 30
Class/Experience Level: Professional
Participant Selection: 10 employees are picked out from 3 companies, they were led through a workshop and then an interview was performed
Data Collection Method: Observation, Survey


This study aims to determine the effects of two continuous integration modeling techniques: Automated Software Integration Flows (ASIF) and Continuous Integration Visualization Technique (CIViT). While both of these modeling techniques have been proven to improve developer understanding on its own, this research proves that: 1. Modeling is an effective practice in analyzing, designing and communicating continuous integration systems. 2. Modeling techniques specifically designed for the continuous integration domain can effectively be used as complements to one another. 3. It affirms in vivo applicability of these modeling techniques to large scale industrial cases and provides practitioners and researchers with recommendations for their use.

The study had two parts. First a workshop is led by researchers with participants where the participants are trained in both modeling techniques and are asked to first build a ASIF model that reflects their current system. After, they are asked to build a CIViT model that reflects their current system. Using the the CIViT model, they are then asked to build a desired ASIF model. After the workshop, an interview takes places to answer the research questions: 1. How may continuous integration modeling be applied to benefit industry practitioners in their day-to-day work? 2. How can the continuous integration specific modeling techniques of ASIF and CIViT be effectively applied in tandem to complement each other?

Ten participants were chosen from each of the three companies from the the Software Center partners, resulting in thirty participants. Four projects were selected from these three companies and four cases were performed in this study. Observations show that it was a challenge introducing both modeling techniques to the engineers. A much debated topic while create the ASIF model was the test coverage of the system. Researches also noted that when create the Wanted ASIF model, many of the engineers didn't refer to the CIViT model. In other words, the engineers tasked with building and maintaining the integration systems already knew what they would like to improve, and CIViT did not change those perceptions.

During the interviews, most participants responded positively to the use of diagrams. They said that diagrams are a good tool to find common understanding and sparked conversations. The process of creating the diagrams in a group was beneficial to the understanding and improvements of the continuous integration system.