"Jumping Through Hoops": Why do Java Developers Struggle With Cryptography APIs?

Author(s): S. Nadi, S. Kruger, M. Mezini, E. Bodden
Venue: International Conference on Software Engineering
Date: 2016

Type of Experiement: Survey/Multi-Case Study
Class/Experience Level: Other
Participant Selection: Participants were selected through a variety of efforts. Randomly selected GitHub repositories, top 100 Stack Overflow posts involving Java cryptography questions, etc.
Data Collection Method: Observation, Survey, Project Artifact(s)


This paper aims to explore the root cause of why Java developers fail to use cryptography libraries successfully. The results are the results of four separate studies 1) an analysis of the top 100 Java cryptography questions on Stack Overflow, 2) a pilot survey directed at 11 developers who asked Java cryptography questions on Stack Overflow 3) 100 randomly selected GitHub code projects that used Java cryptography libraries and were analyzed, and 4) a survey of 37 developers who use Java cryptography APIs. The major findings are described below. The first study reports that there are many people that lack the appropriate domain knowledge to use these APIs correctly. However, there is a significant amount of developers (57%) that report having some domain specific knowledge but the API appears too complex or "unclear underlying implementation prevents them from using the correct sequence of calls & parameters." The most common tasks asked of surveyed developers involved handling the storage and authentication of user login. Developers reported the majority of their time was exploring online resources and stated that lack of documentation and lack of higher level abstractions were their biggest obstacles. In summary, many developers state that the Java cryptography APIs lack proper documentation, and while they may understand the proper cryptographic concepts, they do not understand how to properly utilize the API to achieve the desired result.