The macaque IT cortex but not current artificial vision networks encode object position in perceptually aligned coordinates

Abstract

Efficient interaction with the visual world requires not only accurate object identification but also precise localization of objects in space. While spatial ("where") processing has traditionally been attributed to dorsal stream pathways, recent work has shown that object position can also be decoded from responses in ventral stream areas such as the inferior temporal (IT) cortex. However, because object position in these paradigms is tightly coupled to pixel-based location, it remains unclear whether ventral stream position signals reflect perceptually meaningful spatial representations or simply inherited retinotopic structure. To address this question, we used the motion aftereffect, a classic visual illusion that shifts perceived object position without changing retinal input. Combining large-scale intracortical recordings in macaque IT with matched human psychophysics, we found that motion adaptation induces systematic direction-opponent biases in IT population codes for object position that mirror human perceptual reports, despite identical pixel-level stimuli. These effects are accompanied by adaptation-driven changes in the geometry of IT population representations. We further tested whether artificial vision systems exhibit similar dynamics. Standard feedforward, recurrent, and state-of-the-art video-based neural networks accurately encode object position but fail to produce adaptation-induced position shifts. However, applying empirically derived transformations based on IT adaptation dynamics to model feature spaces is sufficient to generate similar biases. Together, these results indicate that IT represents object position in perceptually aligned coordinates and also highlight a gap between biological and artificial vision systems in capturing history-dependent spatial coding.