Sensor array and camera fusion via unbalanced optimal transport for 3D source localization

Abstract

We address the problem of localizing multiple sources in 3D by combining sensor array measurements with camera observations. We propose a fusion framework extending the covariance matrix fitting method with an unbalanced optimal transport regularization term that softly aligns sensor array responses with visual priors while allowing flexibility in mass allocation. To solve the resulting largescale problem, we adopt a greedy coordinate descent algorithm that efficiently updates the transport plan. Its computational efficiency makes full 3D localization feasible in practice. The proposed framework is modular and does not rely on labeled data or training, in contrast with deep learning-based fusion approaches. Although validated here on acoustic arrays, the method is general to arbitrary sensor arrays. Experiments on real data show that the proposed approach improves localization accuracy compared to sensor-only baselines.