arXiv:2602.08800v1 [cs.OS] 9 Feb 2026.
Kaiyang Zhao, Neha Gholkar*, Hasan Maruf*, Abhishek Dhanotia*, Johannes Weiner*, Gregory Price*, Ning Sun*, Bhavya Dwivedi*, Stuart Clark*, Dimitrios Skarlatos
Carnegie Mellon University
*Meta
Memory is becoming an ever-larger portion of system cost and power in datacenters. Memory expansion via Compute Express Link (CXL) provides an effective way to provide additional memory at lower cost and power. However, making effective use of CXL memory requires software-level memory tiering support for most hyperscaler workloads. Many tiering solutions have been proposed. However, our observations from deploying CXL-enabled servers across a wide range of workloads at a hyperscaler reveal fundamental limitations in both prior approaches and the current Linux support. First, existing mechanisms lack multi-tenancy support, failing to robustly handle either stacked homogeneous or heterogeneous workloads. Second, their control planes offer limited flexibility in balancing memory across tiers, leading to fairness violations and uncontrolled performance variability. Finally, current systems provide insufficient observability, depriving operators of the visibility needed to diagnose performance pathologies at scale. In this paper, we present Equilibria, the first operating system framework that directly tackles these challenges and enables fair, multi-tenant CXL memory tiering at datacenter scale. Equilibria provides per-container controls for memory fairshare allocation and fine-grained observability of tiered-memory usage and operations. It further enforces flexible, user-specified fairness policies through regulated promotion and demotion, and mitigates noisy-neighbor interference by suppressing thrashing. We evaluate Equilibria in the fleet of a large hyperscaler using both production workloads and industry-standard benchmarks. Our results show that Equilibria consistently helps workloads meet their service level objectives (SLOs) while avoiding performance interference. Overall, Equilibria improves performance over the state-of-the-art Linux solution, TPP, by up to 52% and 1.7× for production workloads and benchmarks, respectively. All Equilibria patches have been released to the Linux community.
FULL PAPER: pdf