An Empirical Comparison of EvoSuite and DSpot for Improving Developer-Written Test Suites with Respect to Mutation Score.

Since software faults are usually unknown, researchers and developers rely on mutation analysis i.e., seeding artificial defects, called mutants to measure the quality of their test suites. One aim of test amplification techniques is to improve developer-written test cases so that they kill more mutants and potentially find more real faults. However, these tools tend to be limited in the types of changes and improvements they can make to tests, while also receiving little guidance to tests that kill new mutants. Alternatively, a tool like EvoSuite can generate tests with the benefit of detailed fitness information and have the benefit of more flexibility in terms of evolving a test's structure. However, the process is typically not based on developer-written tests, and consequently, the resulting test suites are less likely to be accepted by human developers. In this paper, we propose modifications to EvoSuite, in a technique we refer to as EvoSuite(Amp), which starts with developer-written tests as seeds, and then aims to evolve these tests in the direction of killing further mutants. We then empirically compare EvoSuite(Amp) with a state-of-the-art test amplification tool, DSpot, on 42 versions of 29 different classes from the Defects4J benchmark, using the original developer-written test suites for each class as the starting point for test generation. In total, EvoSuite(Amp) achieves a statistically better mutation score for 35 of these 42 versions when compared to DSpot.