Near to Far: An Evaluation of Disaggregated Memory for In-Memory Data Processing

Efficient in-memory data processing relies on the availability of sufficient resources, be it CPU time or available main memory. Traditional approaches are coping with resource limitations by either adding more processors or RAM sticks to a single server (scale-up) or by adding multiple servers to a network cluster (scale-out). Further, the InfiniBand interconnect enables Remote Direct Memory Access (RDMA) and thus enhances the possibilities of resource sharing between distinct servers. Resource disaggregation means the (dynamic) sharing of available hardware, e. g., through the network. This paradigm is now further enhanced by the specification of Compute Express Link (CXL). In this paper, we systematically evaluate the implications of memory expansion as a form of resource disaggregation from the perspective of in-memory data processing through the local Ultrapath Interconnect (UPI), RDMA via InfiniBand, and PCIe attached memory via CXL. Our results show that CXL yields behavior that is comparable to UPI and outperforms the inherently asynchronous RDMA connection. Further, we found that handling UPI-attached memory as a type of disaggregated resource can yield additional performance benefits.