arXiv subjects

Yiyang Huang

Publications and source records attributed to Yiyang Huang.

28 records · Page 2Linked to original sources

arXiv2025

D-CoDe: Scaling Image-Pretrained VLMs to Video via Dynamic Compression and Question Decomposition

Video large language models (Vid-LLMs), which excel in diverse video-language tasks, can be effectively constructed by adapting image-pretrained vision-language models (VLMs). However, this adaptation remains challenging, as it requires processing dense and temporally extended visual inputs that exceed the capacity of image-based models. This paper identifies the perception bottleneck and token overload as key challenges in extending image-based VLMs to the video domain. To address these issues, we propose D-CoDe, a training-free adaptation framework that incorporates dynamic compression and question decomposition. Specifically, dynamic compression alleviates the perception bottleneck through adaptive selection of representative frames and content-aware aggregation of spatial tokens, thereby reducing redundancy while preserving informative content. In parallel, question decomposition mitigates token overload by reformulating the original query into sub-questions, guiding the model to focus on distinct aspects of the video and enabling more comprehensive understanding. Experiments demonstrate that D-CoDe effectively improves video understanding across various benchmarks. Furthermore, strong performance on the challenging long-video benchmark highlights the potential of D-CoDe in handling complex video-language tasks. Code is available at https://github.com/hukcc/D-CoDe.

cs.CV↗

arXiv2025

Waltz: Temperature-Aware Cooperative Compression for High-Performance Compression-Based CSDs

Data compression is widely adopted for modern solid-state drives (SSDs) to mitigate both storage capacity and SSD lifetime issues. Researchers have proposed compression schemes at different system layers, including device-side solutions like CCSDs ( c ompression-based c omputational SSDs) and compression supported by host-side, like F2FS (flash-friendly file system). We conduct quantitative studies to understand how host-side and device-side compression schemes affect the temperature and performance of SSD-based storage systems. From our experiments, device-side compression, facilitated by a hardware compression engine, can raise the temperature of CCSDs to intolerable levels, resulting in throttling and service shutdown. In contrast, host-side compression causes software-stack overhead, which often results in large performance degradation and resource consumption. To ensure efficient data compression with high performance and better temperature control, we propose Waltz, a temperature-aware cooperative compression method that schedules (de)compression tasks at the host and device sides by monitoring device temperature. Furthermore, we introduce two variants (Waltzs and Waltzp) for space and performance optimization, respectively. Waltz is implemented within F2FS, achieving high performance while extending SSD lifetime and preventing overheating-induced in-flight shutdowns.

cs.PF↗

arXiv2025

ANNIE: Be Careful of Your Robots

The integration of vision-language-action (VLA) models into embodied AI (EAI) robots is rapidly advancing their ability to perform complex, long-horizon tasks in humancentric environments. However, EAI systems introduce critical security risks: a compromised VLA model can directly translate adversarial perturbations on sensory input into unsafe physical actions. Traditional safety definitions and methodologies from the machine learning community are no longer sufficient. EAI systems raise new questions, such as what constitutes safety, how to measure it, and how to design effective attack and defense mechanisms in physically grounded, interactive settings. In this work, we present the first systematic study of adversarial safety attacks on embodied AI systems, grounded in ISO standards for human-robot interactions. We (1) formalize a principled taxonomy of safety violations (critical, dangerous, risky) based on physical constraints such as separation distance, velocity, and collision boundaries; (2) introduce ANNIEBench, a benchmark of nine safety-critical scenarios with 2,400 video-action sequences for evaluating embodied safety; and (3) ANNIE-Attack, a task-aware adversarial framework with an attack leader model that decomposes long-horizon goals into frame-level perturbations. Our evaluation across representative EAI models shows attack success rates exceeding 50% across all safety categories. We further demonstrate sparse and adaptive attack strategies and validate the real-world impact through physical robot experiments. These results expose a previously underexplored but highly consequential attack surface in embodied AI systems, highlighting the urgent need for security-driven defenses in the physical AI era. Code is available at https://github.com/RLCLab/Annie.

cs.AI↗

arXiv2025

DaDu-Corki: Algorithm-Architecture Co-Design for Embodied AI-powered Robotic Manipulation

Embodied AI robots have the potential to fundamentally improve the way human beings live and manufacture. Continued progress in the burgeoning field of using large language models to control robots depends critically on an efficient computing substrate, and this trend is strongly evident in manipulation tasks. In particular, today's computing systems for embodied AI robots for manipulation tasks are designed purely based on the interest of algorithm developers, where robot actions are divided into a discrete frame basis. Such an execution pipeline creates high latency and energy consumption. This paper proposes \textsc{Corki}\xspace, an algorithm-architecture co-design framework for real-time embodied AI-powered robotic manipulation applications. We aim to decouple LLM inference, robotic control, and data communication in the embodied AI robots' compute pipeline. Instead of predicting action for one single frame, \textsc{Corki}\xspace predicts the trajectory for the near future to reduce the frequency of LLM inference. The algorithm is coupled with a hardware that accelerates transforming trajectory into actual torque signals used to control robots and an execution pipeline that parallels data communication with computation. \textsc{Corki}\xspace largely reduces LLM inference frequency by up to $5.1\times$, resulting in up to $5.9\times$ speed up. The success rate improvement can be up to 13.9\%.

cs.AR↗

arXiv2024

RoboUniView: Visual-Language Model with Unified View Representation for Robotic Manipulation

Utilizing Vision-Language Models (VLMs) for robotic manipulation represents a novel paradigm, aiming to enhance the model's ability to generalize to new objects and instructions. However, due to variations in camera specifications and mounting positions, existing methods exhibit significant performance disparities across different robotic platforms. To address this challenge, we propose RoboUniView in this paper, an innovative approach that decouples visual feature extraction from action learning. We first learn a unified view representation from multi-perspective views by pre-training on readily accessible data, and then derive actions from this unified view representation to control robotic manipulation. This unified view representation more accurately mirrors the physical world and is not constrained by the robotic platform's camera parameters. Thanks to this methodology, we achieve state-of-the-art performance on the demanding CALVIN benchmark, enhancing the success rate in the $D \to D$ setting from 93.0% to 96.2%, and in the $ABC \to D$ setting from 92.2% to 94.2%. Moreover, our model exhibits outstanding adaptability and flexibility: it maintains high performance under unseen camera parameters, can utilize multiple datasets with varying camera parameters, and is capable of joint cross-task learning across datasets. Code is provided for re-implementation. https://github.com/liufanfanlff/RoboUniview

cs.RO↗

arXiv2023

Tight Bounds for Machine Unlearning via Differential Privacy

We consider the formulation of "machine unlearning" of Sekhari, Acharya, Kamath, and Suresh (NeurIPS 2021), which formalizes the so-called "right to be forgotten" by requiring that a trained model, upon request, should be able to "unlearn" a number of points from the training data, as if they had never been included in the first place. Sekhari et al. established some positive and negative results about the number of data points that can be successfully unlearnt by a trained model without impacting the model's accuracy (the "deletion capacity"), showing that machine unlearning could be achieved by using differentially private (DP) algorithms. However, their results left open a gap between upper and lower bounds on the deletion capacity of these algorithms: our work fully closes this gap, obtaining tight bounds on the deletion capacity achievable by DP-based machine unlearning algorithms.

cs.LG↗

arXiv2022

Lemmas of Differential Privacy

We aim to collect buried lemmas that are useful for proofs. In particular, we try to provide self-contained proofs for those lemmas and categorise them according to their usage.

cs.DS↗

arXiv2022

Invertible Mask Network for Face Privacy-Preserving

Face privacy-preserving is one of the hotspots that arises dramatic interests of research. However, the existing face privacy-preserving methods aim at causing the missing of semantic information of face and cannot preserve the reusability of original facial information. To achieve the naturalness of the processed face and the recoverability of the original protected face, this paper proposes face privacy-preserving method based on Invertible "Mask" Network (IMN). In IMN, we introduce a Mask-net to generate "Mask" face firstly. Then, put the "Mask" face onto the protected face and generate the masked face, in which the masked face is indistinguishable from "Mask" face. Finally, "Mask" face can be put off from the masked face and obtain the recovered face to the authorized users, in which the recovered face is visually indistinguishable from the protected face. The experimental results show that the proposed method can not only effectively protect the privacy of the protected face, but also almost perfectly recover the protected face from the masked face.

cs.CV↗

arXiv2020

A novel direct structured-light inspection technique for contaminant and defect detection

The Direct Structured-Light Inspection Technique (DSIT) proposed in this paper is a novel method that can be implemented under two types of binary structured light illumination to detect contaminants and defects on specular surfaces and transparent objects, in which light reflection system is used to detect specular surfaces, while light transmission system is applied for transparent object inspection. Based on this technique, contaminant and defect distribution can be directly obtained without any calculation process. Relevant simulations and experiments are performed to prove the effectiveness of DSIT.

eess.IV↗

arXiv2020

Improved mathematical models of structured-light modulation analysis technique for contaminant and defect detection

Surface quality inspection of optical components is critical in optical and electronic industries. Structured-Light Modulation Analysis Technique (SMAT) is a novel method recently proposed for the contaminant and defect detection of specular surfaces and transparent objects, and this approach was verified to be effective in eliminating ambient light. The mechanisms and mathematical models of SMAT were analyzed and established based on the theory of photometry and the optical characteristics of contaminants and defects. However, there are still some phenomena exist as conundrums in actual detection process, which cannot be well explained. In order to better analyze the phenomena in practical circumstances, improved mathematical models of SMAT are constructed based on the surface topography of contaminants and defects in this paper. These mathematical models can be used as tools for analyzing various contaminants and defects in different systems, and provide effective instruction for subsequent work. Simulations and experiments on the modulation and the luminous flux of fringe patterns have been implemented to verify the validity of these mathematical models. In adddition, by using the fringe patterns with mutually perpendicular sinusoidal directions, two obtained modulation images can be merged to solve the incomplete information acquisition issue caused by the differentiated response of modulation.

physics.optics↗