arXiv subjects

Jan Liphardt

Publications and source records attributed to Jan Liphardt.

3 recordsLinked to original sources

arXiv2026

SocioGesture: Real-Time and Adaptive Social Gesture Perception for Human-Robot Interaction

Robots interacting with people must recognize not only explicit commands, but also social cues such as invitations, refusals, and unavailability. In real deployments, these cues must be inferred from noisy onboard perception under partial occlusion, changing viewpoints, and strict latency constraints. We present SocioGesture, a real-time adaptive social gesture perception system for human-robot interaction (HRI). SocioGesture uses a compact confidence-aware body-hand skeleton representation and a lightweight dual-stream model that fuses body motion with hand articulation for low-latency onboard recognition. To improve deployment robustness, we train the model with occlusion-aware skeleton corruption, exposing it to missing hands, occluded arms, and temporally unstable keypoints without increasing the inference cost. On a social gesture dataset collected in mixed indoor-outdoor HRI scenarios, SocioGesture achieves strong held-out-subject recognition, substantially improves robustness under structured joint occlusion, and runs in real time on a robot-mounted edge device. During deployment, uncertain interaction segments are saved for offline labeling and adaptation, enabling SocioGesture to expand its gesture vocabulary while preserving performance in the original classes. These results demonstrate a practical path toward robust, efficient, and adaptive social perception for interactive robots.

cs.RO↗

arXiv2024

A Paragraph is All It Takes: Rich Robot Behaviors from Interacting, Trusted LLMs

Large Language Models (LLMs) are compact representations of all public knowledge of our physical environment and animal and human behaviors. The application of LLMs to robotics may offer a path to highly capable robots that perform well across most human tasks with limited or even zero tuning. Aside from increasingly sophisticated reasoning and task planning, networks of (suitably designed) LLMs offer ease of upgrading capabilities and allow humans to directly observe the robot's thinking. Here we explore the advantages, limitations, and particularities of using LLMs to control physical robots. The basic system consists of four LLMs communicating via a human language data bus implemented via web sockets and ROS2 message passing. Surprisingly, rich robot behaviors and good performance across different tasks could be achieved despite the robot's data fusion cycle running at only 1Hz and the central data bus running at the extremely limited rates of the human brain, of around 40 bits/s. The use of natural language for inter-LLM communication allowed the robot's reasoning and decision making to be directly observed by humans and made it trivial to bias the system's behavior with sets of rules written in plain English. These rules were immutably written into Ethereum, a global, public, and censorship resistant Turing-complete computer. We suggest that by using natural language as the data bus among interacting AIs, and immutable public ledgers to store behavior constraints, it is possible to build robots that combine unexpectedly rich performance, upgradability, and durable alignment with humans.

cs.RO↗

arXiv2009

Mechanical conversion of low-affinity Integration Host Factor binding sites into high-affinity sites

Although DNA is often bent in vivo, it is unclear how DNA-bending forces modulate DNA-protein binding affinity. Here, we report how a range of DNA-bending forces modulates the binding of the Integration Host Factor (IHF) protein to various DNAs. Using solution fluorimetry and electrophoretic mobility shift assays, we measured the affinity of IHF for DNAs with different bending forces and sequence mutations. Bending force was adjusted by varying the fraction of double-stranded DNA in a circular substrate, or by changing the overall size of the circle (1). DNA constructs contained a pair of Forster Resonance Energy Transfer dyes that served as probes for affinity assays, and read out bending forces measured by optical force sensors (2). Small bending forces significantly increased binding affinity; this effect saturated beyond ~3 pN. Surprisingly, when DNA sequences that bound IHF only weakly were mechanically bent by circularization, they bound IHF more tightly than the linear "high-affinity" binding sequence. These findings demonstrate that small bending forces can greatly augment binding at sites that deviate from a protein's consensus binding sequence. Since cellular DNA is subject to mechanical deformation and condensation, affinities of architectural proteins determined in vitro using short linear DNAs may not reflect in vivo affinities.

q-bio.BM↗