Lightweight, Practical Encrypted Face Recognition with GPU Support
Face recognition typically operates in a client-server setting, where the client extracts a compact face embedding and the server performs similarity search over a template database. Since facial data is highly sensitive, this raises significant privacy concerns. Fully homomorphic encryption (FHE) addresses these concerns by enabling end-to-end encrypted similarity search. However, existing FHE-based protocols are computationally costly and, especially, impose high memory overhead due to large rotation-key sets and bandwidth-bound homomorphic operations. Building on prior work, HyDia (PoPETS 2025), we introduce algorithmic and system-level improvements targeting real-world deployment with resource-constrained (edge) clients. First, we propose BSGS-Diagonal, a fast and memory-efficient similarity computation algorithm that applies a Baby-Step/Giant-Step strategy with precomputed rotations reused across consecutive matrix--vector products. This yields a 91% reduction in rotation keys (~14GB less client memory) and cuts peak server-side CPU RAM usage from over 33GB to 11GB for databases up to 1M entries, with runtime improvements of up to 1.57x for membership verification and 1.43x for identification. Second, we introduce GPU-optimized similarity computation kernels, including an efficient homomorphic Chebyshev evaluator built upon FIDESlib (ISPASS 2025), a CKKS-level GPU library based on OpenFHE. Rather than offloading individual CKKS primitives, our integrated kernels fuse operations to avoid repeated CPU--GPU ciphertext movement and costly FIDESlib/OpenFHE data-structure conversions. Our HyDia and BSGS GPU results achieve up to 9x and 21x speedups on single GPU (and up to 287x and 211x using multi-GPUs), respectively, enabling sub-second encrypted face recognition for databases up to 2^16 entries (or 2^19 entries in a multi-GPU setting), while further reducing host memory usage.