Impact of API change and fault-proneness impact on user ratings of Android apps

Author(s): Gabriele Bavota, Mario Linares-Vasquez, Carlos Eduardo Bernal-Cardenas
Venue: Transactions on Software Engineering
Date: November 2014

Type of Experiement: Survey/Multi-Case Study
Sample Size: 45
Class/Experience Level: Professional
Participant Selection: 45 developers were selected from the apps that they analyzed in the first part of the study
Data Collection Method: Observation, Survey


The objective of this study was to provide strong evidence that solidified the possibility of a relationship between the success of an app (as measured by user ratings) and the frequency with which the API’s used in those apps underwent changes or bug fixes. To prove this two case studies were conducted.

Study number 1 analyzed around 6000 free Android apps to find purely numerical data on the relationship between fault-proneness and depreciation in APIs. It divided these apps up into 30 different categories and were sure to select only those with over 10 ratings that they could access the change history of the API’s that were used. The independent variable was the number of commits in the API code using the terms “fix” or “issue” and the dependent variable was the average rating. Using this data they split these apps into groups with “high”, “medium”, and “low” scores and found that highly rated apps had only 4.4 issues and underwent 25 changes on average while poorly rated apps had an average of 24.5 bugs and underwent 37 changes.

The second study was a survey completed by 45 professional developers who had worked on the apps analyzed in the first study. One of the questions asked which of 4 different causes was most likely to lead to poor user ratings, and app crashed and bugs was selected to be the first choice. Following up on that topic, developers were asked which categories caused frequent bugs that led to the application to crash or interfere with the user’s work flow and 71% of developers selected deprecation and bugs in the APIs that their app used.

Ultimately, the study concluded that there is a relationship between how often and API changes and how many bug fixes are found and the overall user rating of the app. Their solution to this problem is for APIs to openly list their change history so developers can make a better informed decision when selecting which APIs to use in their app.