How Surveys, Tutors, and Software Help to Assess Scum Adoption in a Classroom Software Engineering Project

Author(s): Christop Matthies, Thomas Kowark, Keven Richly, Matthias Uflacker, Hasso Plattner
Venue: 38th IEEE International Conference on Software Engineering
Date: 2016

Type of Experiement: Survey/Multi-Case Study
Sample Size: 38
Class/Experience Level: Undergraduate Student
Participant Selection: Final Year Undergraduate Class
Data Collection Method: Observation, Survey, Code Metric, Project Artifact(s)


This study looked at the adoption and implementation of Scrum within an educational environment. The goal was to understand how students implemented Scrum, and what made good indicators to whether or not a team was implementing Scrum correctly. The researchers analyzed 38, Senior-level, undergraduate students, working together to build a software system. These students were placed on teams and required to use a scaled version of Scrum. The three techniques the researchers used to measure Scrum implementation were periodic surveys, tutor accompanied team meetings and ScrumLint for the produced artifacts by each team.

The goal of the surveys was to determine the student's satisfaction with Scrum as a whole, as well as to track the actual implementation of Scrum. While the surveys gave some insight to the overall project, the researchers found that they were not substantive on their own. What the surveys lacked was a concrete link to development artifacts or issues that arose within the team.

The second research technique used was having tutors attend team meetings where they could rate teams based on their use of Scrum. In order to not interfere or bias the teams with information, the tutors were a passive sit-in on the meetings, only interjecting when there were blatant violations of Scrum practices. The researchers found the tutors to be more beneficial than the surveys, as they were able to gather concrete data from each of the teams, while also providing valuable Scrum advice on the spot.

Towards the end of the study, the researchers used ScrumLint. ScrumLint is an automatic "linter" for agile processes developed specifically for this software course. It is used to support development teams in adhering to best Scrum practices. In this case, ScrumLint was able to analyze data from the Github repo used by the class. The tool offers ten conformance metrics such as "developer productivity", "Backlog maintenance" and "XP practices". This tool gave the researchers and teams an objective understanding of how each team was performing.

The combination of techniques used by the researchers allowed for a better understanding of Scrum adoption by the class. In general, they found most teams to be accepting of Scrum and adhering to best practices whenever possible. They also found that the surveys allowed the tutors to immediately curve or fix any possible deviations from Scrum's best practices. One take away the researchers had was to provide future teams with ScrumLint at the beginning of the course. They believe it would be interesting to see how teams use this tool for their own benefit.