(Nothing But) Many Eyes Make All Bugs Shallow

Open source package repositories have become a crucial component of the modern software supply chain since they enable developers to easily and rapidly import code written by others. However, low quality, poorly vetted code residing in such repositories exposes developers and end-users to dangerous bugs and vulnerabilities at a large scale. Such issues have recently led to the creation of government-backed verification standards pertaining to packages, as well as a significant body of developer folklore regarding what constitutes a reliable package. However, there exists little academic research assessing the relationships between recommended development practices and known package issues in this domain. Motivated by this gap in understanding, we conduct a large-scale study that formally evaluates whether adherence to these guidelines meaningfully impacts reported issues and bug maintenance activity across the most widely utilized npm packages (encompassing 7,162 packages with over 100K weekly downloads each), which unveiled wide disparities across package-level metrics. We find that it is only recommendations pertaining to a broad notion of scrutiny that provide strong and reliable insights into the reporting and resolving of package issues. These findings pose significant implications for developers, who seek to identify well-maintained packages for use, as well as security researchers, who seek to identify suspicious packages for critical observation.