First-generation Memory Disaggregation for Cloud Platforms

In Azure, up to 25% of memory is stranded, i.e., it is leftover after the servers’ cores have been rented to VMs. Memory disaggregation promises to reduce this stranding. However, making disaggregation practical for production cloud deployment remains challenging. For example, RDMA-based disaggregation involves too much overhead for common workloads and transparent latency management is incompatible with virtualization acceleration. The emerging Compute Express Link (CXL) standard offers a low-overhead substrate to build memory disaggregation while overcoming these challenges. This paper proposes a first-generation CXL-based disaggregation system that meets the requirements of cloud providers. Our system includes a memory pool controller, and prediction-based system software and distributed control plane designs. Its predictions of VM latency sensitivity and memory usage allow it to split workloads across local and pooled memory while mitigating the higher pool latency. Our analysis of production clusters shows that small pools of 8-32 sockets are sufficient to reduce stranding significantly. It also shows that ∼50% of all VMs never touch 50% of their rented memory. In emulated experiments with 150+ workloads, we show our pooling approach incurs a configurable performance loss between 1-5%. Finally, we show that disaggregation can achieve a 9-10% reduction in overall DRAM, which represents hundreds of millions of dollars in cost savings for a large cloud provider.