ProactivePIM: Accelerating Weight-Sharing Embedding Layer with PIM for Scalable Recommendation System

Abstract

Although deep learning-based personalized recommendation systems provide qualified recommendations, they strain data center resources. The main bottleneck is the embedding layer, which is highly memory-intensive due to its sparse, irregular access patterns to embeddings. Recent near-memory processing (NMP) and processing-in-memory (PIM) architectures have addressed these issues by exploiting parallelism within memory. However, as model sizes increase year by year and can exceed server capacity, inference on single-node servers becomes challenging, necessitating the integration of model compression. Various algorithms have been proposed for model size reduction, but they come at the cost of increased memory access and CPU-PIM communication. We present ProactivePIM, a PIM system tailored for weight-sharing algorithms, a family of compression methods that decompose an embedding table into compact subtables, such as QR-trick and TT-Rec. Our analysis shows that embedding layer execution with weight-sharing algorithms increases memory access and incurs CPU-PIM communication. We also find that these algorithms exhibit unique data locality characteristics, which we name intra-GnR locality. ProactivePIM accelerates weight-sharing algorithms by utilizing a heterogeneous HBM-DIMM memory architecture with integration of a two-level PIM system of base-die PIM (bd-PIM) and bank-group PIM (bg-PIM) inside the HBM. To gain further speedup, ProactivePIM prefetches embeddings with high intra-GnR locality into an SRAM cache within bg-PIM and eliminates the CPU-PIM communication through duplication of target subtables across bank groups. With additional optimization techniques, our design effectively accelerates weight-sharing algorithms, achieving 2.22x and 2.15x speedup in QR-trick and TT-Rec, respectively, compared to the baseline architecture.