Automatic Test Image Generation using Procedural Noise

Author(s): Matthew Patrick, Matthew D. Castle, Richard O. J. H. Stutt, Christopher A. Gilligan
Venue: 2016 31st IEEE/ACM International Conference on Automated Software Engineering (ASE)
Date: 2016

Type of Experiement: Case Study
Sample Size: 3
Class/Experience Level: Other
Participant Selection: Alternative approaches to generate test images.
Data Collection Method: Code Metric


The authors of this research paper noticed that although source code has been thoroughly tested and testing processes have been significantly automated, little has been done to test programs that deal with images. This is particularly important for applications in the medical field, where the images produced by software may be used to make life-critical decisions. Test images must be generated that can be compared to these programs’ output, but the large dimensionality (number of pixel values) of these images can make them difficult to produce. The authors of this paper propose using Perlin Noise along with the Moran’s I spatial statistic and evolution via a genetic algorithm in order to generate test images that are more precise (can generate images with specific properties), efficient (time it takes to generate an image), flexible (works with new images as well as existing ones), and scalable (able to work with a range of image sizes) than previous approaches.

To test their approach, the authors utilize a technique known as metamorphic testing. Since it is difficult to know what the correct output should be without using the software itself to find out, metamorphic testing instead uses information that determines how the output should change when the inputs are adjusted. Using this technique, the authors tested their approach on a simulation of the spread of the Oriental Chestnut Gall Wasp in Great Britain. They generated images using three techniques: Random noise, Poisson Cluster Process (clusters of points with Poisson distributions), and Perlin noise. They found that their method was able to produce images that matched the target properties more accurately than previous approaches, in less time, and that it could identify faults in the software more effectively.