A bug finder refined by a large set of open-source projects

Context Static bug detection techniques are commonly used to automatically detect software bugs. The biggest obstacle to the wider adoption of static bug detection tools is false positives, i.e., reported bugs that developers do not have to act on. Objective The objective of this study is to reduce false positives resulting from static bug detection tools and to detect new bugs by exploring the effectiveness of a feedback-based bug detection rule design. Method We explored a large number of software projects and applied an iterative feedback-based process to design bug detection rules. The outcome of the process is a set of ten bug detection rules, which we used to build a feedback-based bug finder, FeeFin. Specifically, we manually examined 1622 patches to identify bugs and fix patterns, and implement bug detection rules. Then, we refined the rules by repeatedly using feedback from a large number of software projects. Results We applied FeeFin to the latest versions of the 1880 projects on GitHub to detect previously unknown bugs. FeeFin detected 98 new bugs, 63 of which have been reviewed by developers: 57 were confirmed as true bugs, and 9 were confirmed as false positives. In addition, we investigated the benefits of our FeeFin process in terms of new and improved bug patterns. We verified our bug patterns with four existing tools, namely PMD, FindBugs, Facebook Infer, and Google Error Prone, and found that our FeeFin process has the potential to identify new bug patterns and also to improve existing bug patterns. Conclusion Based on the results, we suggest that static bug detection tool designers identify new bug patterns by mining real-world patches from a large number of software projects. In addition, the FeeFin process is helpful in mitigating false positives generated from existing tools by refining their bug detection rules.