Do Developers Discuss Design?

Author(s): João Brunet, Gail C. Murphy, Ricardo Terra, Jorge Figueiredo, Dalton Serey
Venue: International Conference on Software Engineering
Date: 2014

Type of Experiement: Case Study
Sample Size: 77
Class/Experience Level: Other
Participant Selection: Most popular projects on GitHub


In this article the authors discuss a study they performed to determine if developers, specifically in open source projects, discuss software design with each other. After looking at a sample of projects on GitHub they found that 68% of projects did not include any formal documentation of design for the project. This lead to their research of looking through comments, commits, issues and other informal discussion to see if the developers were conducting their design decisions in another way.

In designing their study two research questions were defined.
RQ1: To what extent do developers discuss design in open-source projects?
RQ2: Which developers discuss design?

Next they developed and evaluated “a prototype based on machine learning technique to automatically identify design discussions”. This prototype was tested against human analyzed responses and given training data to improve its accuracy.

After the researchers were confidant in the classifier they ran it against all 102,122 discussions in their data set (the data set consisted of top projects from GitHub). With this data they answered the research questions as follows.
RQ1: Of all of the analyzed discussions 25% of them were determined to be design discussions. Design discussions are things that discussed design patterns, dependencies, proper coding techniques (i.e. not finding hacked solutions) but not things like code style guidelines (i.e. spacing, naming conventions etc), or other trivial issues.
RQ2: Only 36% of developers that contributed to a project discussed design. To figure out which developers contributed their “coverage” was calculated. Coverage is defined as “the proportion of all design discussions in a project to which a developer has contributed”. Given the developers coverage it was found that 99% of developers contribute to less than 15% of design decisions, meaning that very few developers contribute to a broad range of design discussions.

Overall the study was able to begin to analyze design communication in the open source development community. It showed that design is discussed informally by a moderate number of developers and that there may not be a central group of developers who contribute to a broad range of design decisions.