arXiv subjects

Jianning Deng

Publications and source records attributed to Jianning Deng.

4 recordsLinked to original sources

arXiv2026

PAOLI: Pose-free Articulated Object Learning from Sparse-view Images

We present a method for modeling articulated objects from sparse images with unknown camera poses. Existing approaches require dense multi-view observations and ground-truth camera poses shared across articulation states, whereas our approach operates with as few as four unposed views per articulation state, without requiring ground-truth poses or shared cross-state calibration. The key challenge is that sparse-view reconstruction can recover each articulation state independently, but does not provide the cross-state correspondences required for part segmentation and kinematic reasoning. We address this by optimizing a dense deformation field that aligns independently reconstructed states and establishes dense correspondences across them. An iterative robust estimation procedure then separates static and moving parts and recovers their kinematic parameters. Finally, we refine geometry, appearance, and motion using self-supervised losses that enforce cross-view and cross-state consistency. Experiments on standard benchmarks and real-world examples show that our method produces accurate articulated object representations under substantially weaker input assumptions than prior work.

cs.CV↗

arXiv2024

Articulate your NeRF: Unsupervised articulated object modeling via conditional view synthesis

We propose a novel unsupervised method to learn the pose and part-segmentation of articulated objects with rigid parts. Given two observations of an object in different articulation states, our method learns the geometry and appearance of object parts by using an implicit model from the first observation, distils the part segmentation and articulation from the second observation while rendering the latter observation. Additionally, to tackle the complexities in the joint optimization of part segmentation and articulation, we propose a voxel grid-based initialization strategy and a decoupled optimization procedure. Compared to the prior unsupervised work, our model obtains significantly better performance, and generalizes to objects with multiple parts while it can be efficiently from few views for the latter observation.

cs.CV↗

arXiv2024

Robust 3D Object Detection from LiDAR-Radar Point Clouds via Cross-Modal Feature Augmentation

This paper presents a novel framework for robust 3D object detection from point clouds via cross-modal hallucination. Our proposed approach is agnostic to either hallucination direction between LiDAR and 4D radar. We introduce multiple alignments on both spatial and feature levels to achieve simultaneous backbone refinement and hallucination generation. Specifically, spatial alignment is proposed to deal with the geometry discrepancy for better instance matching between LiDAR and radar. The feature alignment step further bridges the intrinsic attribute gap between the sensing modalities and stabilizes the training. The trained object detection models can deal with difficult detection cases better, even though only single-modal data is used as the input during the inference stage. Extensive experiments on the View-of-Delft (VoD) dataset show that our proposed method outperforms the state-of-the-art (SOTA) methods for both radar and LiDAR object detection while maintaining competitive efficiency in runtime. Code is available at https://github.com/DJNing/See_beyond_seeing.

cs.CV↗

arXiv2022

Self-Supervised Scene Flow Estimation with 4-D Automotive Radar

Scene flow allows autonomous vehicles to reason about the arbitrary motion of multiple independent objects which is the key to long-term mobile autonomy. While estimating the scene flow from LiDAR has progressed recently, it remains largely unknown how to estimate the scene flow from a 4-D radar - an increasingly popular automotive sensor for its robustness against adverse weather and lighting conditions. Compared with the LiDAR point clouds, radar data are drastically sparser, noisier and in much lower resolution. Annotated datasets for radar scene flow are also in absence and costly to acquire in the real world. These factors jointly pose the radar scene flow estimation as a challenging problem. This work aims to address the above challenges and estimate scene flow from 4-D radar point clouds by leveraging self-supervised learning. A robust scene flow estimation architecture and three novel losses are bespoken designed to cope with intractable radar data. Real-world experimental results validate that our method is able to robustly estimate the radar scene flow in the wild and effectively supports the downstream task of motion segmentation.

cs.CV↗