Experimental Analysis: A New Paradigm for Program Analysis

Program analysis techniques analyze software systems to gather information about them, which can then be used in various software-engineering related tasks. Traditionally, program analysis techniques have been classified as static or dynamic, depending on whether they operate independent of, or dependent on, specific program executions. Recently, however, researchers have attempted to move beyond this classification and pursue other program analysis approaches. One such approach involves incorporating, into analysis techniques, characteristics of experimentation, such as the sampling of populations, statistical hypothesis testing, design-of-experiments, and manipulation of independent variables, in order to draw inferences about systems and causal relationships. In this paper, we argue that this approach, experimental analysis, forms a new and powerful paradigm by which program analysis …