Lightning in the cloud: a study of very short bottlenecks on n-tierweb application performance

In this paper, we describe an experimental study of very long response time (VLRT) requests in the latency long tail problem. Applying micro-level event analysis on fine-grained measurement data from n-tier application benchmarks, we show that very short bottlenecks (from tens to hundreds of milliseconds) can cause queue overflows that propagate through an n-tier system, resulting in dropped messages and VLRT requests due to timeout and retransmissions. Our study shows that even at moderate CPU utilization levels, very short bottlenecks arise from several system layers, including Java garbage collection, anti-synchrony between workload bursts and DVFS clock rate adjustments, and statistical workload interferences among co-located VMs. As a simple model for a variety of causes of VLRT requests, very short bottlenecks form the basis for a discussion of general remedies for VLRT requests, regardless of their origin. For example, methods that reduce or avoid queue amplification in an n-tier system result in non-trivial trade-offs among system components and their configurations. Our results show interesting challenges remain in both causes and effective remedies of very short bottlenecks.