Documented decision-making strategies and decision knowledge in open source projects: An empirical study on Firefox issue reports.

Empirical study on documented decisions based on 260 Firefox issue discussions.Shows that more naturalistic than rational decision-making was documented.Shows that developers mostly documented knowledge on decision context and solution.Further research required on the impact and support of naturalistic documentation.Our coding tables can be used by other researchers for documentation analysis. Context: Decision-making is a vital task during software development. Typically, issue tracking systems are used to document decisions in large open source projects where developers are spread across the world. While most decision documentation approaches assume that developers use rational decision strategies, in practice also naturalistic strategies are employed. However, quantitative studies of the distribution of decision strategies and related knowledge are missing.Objective: Our overall goal is to provide insights and ideas for further research to systematically support and document decision-making during software development in open source projects. In this paper, we analyze decisions documented in comments to issue reports in order to understand the documentation of decision-making in detail.Method: We coded the comments of 260 issue reports of the open source project Firefox for decision-making strategies and knowledge on decisions. Then, we statistically analyzed the coded data with regard to the dominant decision strategy, the distribution of decision strategies and knowledge, and the relations between strategy and knowledge.Results: The vast majority of documented decision-making strategies was naturalistic. Interestingly, for feature requests the percentage of rational decision-making strategies was higher than for bugs. Documented knowledge mostly concerned the decision context. More solutions were documented together with a higher amount of naturalistic decision-making. However, solutions were negatively correlated with the assessment of the situation. So, developers are likely to exploit and document decision problems and solutions in an imbalanced way.Conclusion: Our analysis revealed important insights on how decision-making and its related knowledge is documented during software development in open source projects. For instance, we found naturalistic decision-making to play an important role for development decisions. Our coding tables can be used by other researchers to further investigate our results. The study insights should be reflected in decision support systems to improve their effectiveness and acceptance by developers.