arXiv subjects

Jeremy Collins

Publications and source records attributed to Jeremy Collins.

3 recordsLinked to original sources

arXiv2026

FloMo: 3D Scene Flow as a World Action Model Intermediate for Learning from Human Video

Pretrained video generation models distill internet-scale motion priors into robot policies, but full video is a redundant signal for action prediction. We argue that dense 3D motion is the medium through which video pretraining best transfers to robot actions, and that predicting motion rather than video affords better sample-efficiency for action prediction. FloMo is a 5B video-generation model fine-tuned to predict 3D scene-flow and action tokens. Rendering scene flow as RGB and encoding it into the same latent space as video enables a pretrained video generation backbone to efficiently adapt to predict 3D scene flow. Co-trained on action-free human video and action-labeled robot teleoperation data, our model benefits from human-video pretraining and generalizes to unseen objects, motion primitives, and tasks. Replacing motion prediction with a video prediction objective degrades out-of-distribution generalization, while adding a video prediction target alongside motion lowers policy success rather than improving it. When motion, video, and action are predicted jointly, action tokens attend far more to motion than to video, suggesting that predicting video wastes model capacity. Together, these results indicate that motion effectively mediates the transfer from video pretraining to action. More information can be found at flomo-wam.github.io.

cs.CV↗

arXiv2026

Less Decoder is More Encoder: Geometric Representation Learning from Novel View Synthesis

This paper examines the role of Novel View Synthesis (NVS) in geometric representation learning. In principle, NVS should reason about 3D scene structure, thereby enabling transferable multi-view geometric representations. Yet, existing encoder-based NVS methods yield poor representations. This is not because of a lack of supervisory signal, but rather due to inconspicuous architectural choices: \textit{spatially expressive decoders} that dilute representational capabilities of the scene encoder, and \textit{low-level pixel-space targets} that hinder feature learning. We present SNAP, a self-supervised encoder-decoder transformer that addresses both through a pose-conditioned local decoder and a latent-space reconstruction objective. SNAP is task agnostic, and we show that it is competitive with special-purpose geometry-supervised methods. SNAP also performs competitively against self-supervised representations across five tasks: visual localization, pose estimation, point correspondence, depth estimation, and robot manipulation. Remarkably, SNAP's patch features exhibit emergent viewpoint invariance that approaches heavily supervised models despite lower compute and data budgets. Under camera shifts where standard 2D representations collapse, SNAP degrades more gracefully, revealing that restricting decoder expressivity actively prevents the suppression of transferable geometric structure. https://snap-nvs.github.io

cs.CV↗

arXiv2021

Tendon-Driven Soft Robotic Gripper for Berry Harvesting

Global berry production and consumption have significantly increased in recent years, coinciding with increased consumer awareness of the health-promoting benefits of berries. Among them, fresh market blackberries and raspberries are primarily harvested by hand to maintain post-harvest quality. However, fresh market berry harvesting is an arduous, costly endeavor that accounts for up to 50% of the worker hours. Additionally, the inconsistent forces applied during hand-harvesting can result in an 85% loss of marketable berries due to red drupelet reversion (RDR). Herein, we present a novel, tendon-driven soft robotic gripper with active contact force feedback control, which leverages the passive compliance of the gripper for the gentle harvesting of blackberries. The versatile gripper was able to apply a desired force as low as 0.5 N with a mean error of 0.046 N, while also holding payloads that produce forces as high as 18 N. Field test results indicate that the gripper is capable of harvesting berries with minimal berry damage, while maintaining a harvesting reliability of 95% and a harvesting rate of approximately 4.8 seconds per berry.

cs.RO↗