arXiv subjects

Bin Zhu

Publications and source records attributed to Bin Zhu.

At least 109 records · Page 6Linked to original sources

arXiv2025

Advancing Food Nutrition Estimation via Visual-Ingredient Feature Fusion

Nutrition estimation is an important component of promoting healthy eating and mitigating diet-related health risks. Despite advances in tasks such as food classification and ingredient recognition, progress in nutrition estimation is limited due to the lack of datasets with nutritional annotations. To address this issue, we introduce FastFood, a dataset with 84,446 images across 908 fast food categories, featuring ingredient and nutritional annotations. In addition, we propose a new model-agnostic Visual-Ingredient Feature Fusion (VIF$^2$) method to enhance nutrition estimation by integrating visual and ingredient features. Ingredient robustness is improved through synonym replacement and resampling strategies during training. The ingredient-aware visual feature fusion module combines ingredient features and visual representation to achieve accurate nutritional prediction. During testing, ingredient predictions are refined using large multimodal models by data augmentation and majority voting. Our experiments on both FastFood and Nutrition5k datasets validate the effectiveness of our proposed method built in different backbones (e.g., Resnet, InceptionV3 and ViT), which demonstrates the importance of ingredient information in nutrition estimation. https://huiyanqi.github.io/fastfood-nutrition-estimation/.

cs.CV↗

arXiv2025

Interplay of $95$ GeV Diphoton Excess and Dark Matter in Supersymmetric Triplet Model

The decay of the Higgs boson and the nature of dark matter remain fundamental challenges in particle physics. We investigate the $95$ GeV diphoton excess and dark matter within the framework of the triplet-extended Minimal Supersymmetric Standard Model (TMSSM). In this model, an additional Hypercharge $Y=0$, $SU(2)_L$ triplet superfield is introduced. Mixing between the triplet and doublet Higgs states enhances the diphoton signal strength of the $95$ GeV Higgs boson, resulting in $μ_{γγ}^{\text{CMS+ATLAS}} = 0.24_{-0.08}^{+0.09}$, which is consistent with experimental observations. This enhancement arises primarily from charged Higgs loop contributions. Additionally, the model accommodates viable dark matter candidates in the form of a bino-dominated neutralino. The relic density is reduced to the observed value through resonance-enhanced annihilation via the Higgs portal or co-annihilation with the triplino or higgsino. This reduction remains consistent with constraints from direct and indirect detection experiments. A comprehensive parameter scan demonstrates that the TMSSM can simultaneously explain the $95$ GeV diphoton excess, the observed $125$ GeV Higgs mass, and the dark matter relic density, establishing a compelling and theoretically consistent framework.

hep-ph↗

arXiv2025

Conditional entropy for Amenable group actions

Let G be an infinite discrete countable amenable group acting continuously on a Lebesgue space X. In this article, using partition and factor-space, the conditional entropy of the action G is defined. We introduction some properties of conditional entropy for amenable group actions and the corresponding decomposition theorem is obtained.

math.DS↗

arXiv2025

Carrollian Amplitudes and Holographic Correlators in AdS3/CFT2

We study Carrollian amplitudes of massless scalars in (1+2) Minkowski space. Using the prescription recently shown by Alday et al. [JHEP 03 (2025) 158] originally designed for the AdS4 Witten diagrams, we show that AdS3 Witten diagrams in position space in the flat space limit reduce to Carrollian amplitudes. The flat space limit in the bulk is implemented by the Carrollian limit at the boundary. Focusing on four-point correlators with contact and exchange diagrams, we show that the Carrollian limit makes the universality of the bulk point singularity manifest upon performing analytic continuation to the Lorentzian signature of the boundary correlators. Unlike four-point Carrollian amplitudes in (1+3) dimensions, the (1+2) dimensional ones are non-distributional, having analytic properties simpler than the AdS correlators. We also observe for the first time a double copy structure of Carrollian amplitudes.

hep-th↗

arXiv2025

Look Before You Decide: Prompting Active Deduction of MLLMs for Assumptive Reasoning

Recently, Multimodal Large Language Models (MLLMs) have achieved significant success across multiple disciplines due to their exceptional instruction-following capabilities and extensive world knowledge. However, whether these MLLMs possess human-like compositional reasoning abilities remains an open problem. To unveil their reasoning behaviors, we first curate a \textbf{M}ultimodal \textbf{A}ssumptive \textbf{R}ea\textbf{s}oning Benchmark (MARS-Bench) in this paper. Interestingly, we find that most prevalent MLLMs can be easily fooled by the introduction of a presupposition into the question, whereas such presuppositions appear naive to human reasoning. Besides, we also propose a simple yet effective method, Active Deduction (AD), a novel reinforcement learning paradigm to encourage the model to actively perform composite deduction before reaching a final decision. Equipped with the proposed AD method, a MLLM demonstrates significant improvements in assumptive reasoning abilities without compromising its general-purpose question-answering performance. We also provide extensive evaluations of both open-source and private MLLMs on MARS-Bench, along with experimental analyses of the AD method.

cs.CV↗

arXiv2025

Efficient Prompt Tuning for Hierarchical Ingredient Recognition

Fine-grained ingredient recognition presents a significant challenge due to the diverse appearances of ingredients, resulting from different cutting and cooking methods. While existing approaches have shown promising results, they still require extensive training costs and focus solely on fine-grained ingredient recognition. In this paper, we address these limitations by introducing an efficient prompt-tuning framework that adapts pretrained visual-language models (VLMs), such as CLIP, to the ingredient recognition task without requiring full model finetuning. Additionally, we introduce three-level ingredient hierarchies to enhance both training performance and evaluation robustness. Specifically, we propose a hierarchical ingredient recognition task, designed to evaluate model performance across different hierarchical levels (e.g., chicken chunks, chicken, meat), capturing recognition capabilities from coarse- to fine-grained categories. Our method leverages hierarchical labels, training prompt-tuned models with both fine-grained and corresponding coarse-grained labels. Experimental results on the VireoFood172 dataset demonstrate the effectiveness of prompt-tuning with hierarchical labels, achieving superior performance. Moreover, the hierarchical ingredient recognition task provides valuable insights into the model's ability to generalize across different levels of ingredient granularity.

cs.MM↗

arXiv2025

Don't Deceive Me: Mitigating Gaslighting through Attention Reallocation in LMMs

Large Multimodal Models (LMMs) have demonstrated remarkable capabilities across a wide range of tasks. However, their vulnerability to user gaslighting-the deliberate use of misleading or contradictory inputs-raises critical concerns about their reliability in real-world applications. In this paper, we address the novel and challenging issue of mitigating the negative impact of negation-based gaslighting on LMMs, where deceptive user statements lead to significant drops in model accuracy. Specifically, we introduce GasEraser, a training-free approach that reallocates attention weights from misleading textual tokens to semantically salient visual regions. By suppressing the influence of "attention sink" tokens and enhancing focus on visually grounded cues, GasEraser significantly improves LMM robustness without requiring retraining or additional supervision. Extensive experimental results demonstrate that GasEraser is effective across several leading open-source LMMs on the GaslightingBench. Notably, for LLaVA-v1.5-7B, GasEraser reduces the misguidance rate by 48.2%, demonstrating its potential for more trustworthy LMMs.

cs.AI↗

arXiv2025

Probing Millicharged Dark Matter with Magnetometer Coupled to Circuit

We present a novel approach to detect millicharged dark matter (mDM) by using a high-sensitivity magnetometer coupled with the resonant and broadband readout circuits. In the external magnetic field, the interaction between mDM and the photon field introduces an effective current corresponding to the mDMs annihilation into photons that produces a faint oscillating magnetic field signal, with a frequency uniquely determined by twice the mDM mass. By calculating the expected signal for two experimental configurations -- toroidal and solenoidal magnetic fields -- we show the potential to explore the uncharted regions of mDM parameter space. Our analysis establishes unprecedented constraints on the mDM coupling constant across the mass range $1\times 10^{-12}~\mathrm{eV}$ to $6 \times 10^{-8}~\mathrm{eV}$, surpassing existing experimental limits by more than ten orders of magnitude.

hep-ph↗

arXiv2025

A Newton Interior-Point Method for $\ell_0$ Factor Analysis

Factor Analysis is an effective way of dimensionality reduction achieved by revealing the low-rank plus sparse structure of the data covariance matrix. The corresponding model identification task is often formulated as an optimization problem with suitable regularizations. In particular, we use the nonconvex discontinuous $\ell_0$ norm in order to induce the sparsity of the covariance matrix of the idiosyncratic noise. This paper shows that such a challenging optimization problem can be approached via an interior-point method with inner-loop Newton iterations. To this end, we first characterize the solutions to the unconstrained $\ell_0$ regularized optimization problem through the $\ell_0$ proximal operator, and demonstrate that local optimality is equivalent to the solution of a stationary-point equation. The latter equation can then be solved using standard Newton's method, and the procedure is integrated into an interior-point algorithm so that inequality constraints of positive semidefiniteness can be handled. Finally, numerical examples validate the effectiveness of our algorithm.

math.OC↗

arXiv2025

HD-EPIC: A Highly-Detailed Egocentric Video Dataset

We present a validation dataset of newly-collected kitchen-based egocentric videos, manually annotated with highly detailed and interconnected ground-truth labels covering: recipe steps, fine-grained actions, ingredients with nutritional values, moving objects, and audio annotations. Importantly, all annotations are grounded in 3D through digital twinning of the scene, fixtures, object locations, and primed with gaze. Footage is collected from unscripted recordings in diverse home environments, making HDEPIC the first dataset collected in-the-wild but with detailed annotations matching those in controlled lab environments. We show the potential of our highly-detailed annotations through a challenging VQA benchmark of 26K questions assessing the capability to recognise recipes, ingredients, nutrition, fine-grained actions, 3D perception, object motion, and gaze direction. The powerful long-context Gemini Pro only achieves 38.5% on this benchmark, showcasing its difficulty and highlighting shortcomings in current VLMs. We additionally assess action recognition, sound recognition, and long-term video-object segmentation on HD-EPIC. HD-EPIC is 41 hours of video in 9 kitchens with digital twins of 413 kitchen fixtures, capturing 69 recipes, 59K fine-grained actions, 51K audio events, 20K object movements and 37K object masks lifted to 3D. On average, we have 263 annotations per minute of our unscripted videos.

cs.CV↗

arXiv2025

Next Patch Prediction for Autoregressive Visual Generation

Autoregressive models, built based on the Next Token Prediction (NTP) paradigm, show great potential in developing a unified framework that integrates both language and vision tasks. Pioneering works introduce NTP to autoregressive visual generation tasks. In this work, we rethink the NTP for autoregressive image generation and extend it to a novel Next Patch Prediction (NPP) paradigm. Our key idea is to group and aggregate image tokens into patch tokens with higher information density. By using patch tokens as a more compact input sequence, the autoregressive model is trained to predict the next patch, significantly reducing computational costs. To further exploit the natural hierarchical structure of image data, we propose a multi-scale coarse-to-fine patch grouping strategy. With this strategy, the training process begins with a large patch size and ends with vanilla NTP where the patch size is 1$\times$1, thus maintaining the original inference process without modifications. Extensive experiments across a diverse range of model sizes demonstrate that NPP could reduce the training cost to around 0.6 times while improving image generation quality by up to 1.0 FID score on the ImageNet 256x256 generation benchmark. Notably, our method retains the original autoregressive model architecture without introducing additional trainable parameters or specifically designing a custom image tokenizer, offering a flexible and plug-and-play solution for enhancing autoregressive visual generation.

cs.CV↗

arXiv2025

SwapAnyone: Consistent and Realistic Video Synthesis for Swapping Any Person into Any Video

Video body-swapping aims to replace the body in an existing video with a new body from arbitrary sources, which has garnered more attention in recent years. Existing methods treat video body-swapping as a composite of multiple tasks instead of an independent task and typically rely on various models to achieve video body-swapping sequentially. However, these methods fail to achieve end-to-end optimization for the video body-swapping which causes issues such as variations in luminance among frames, disorganized occlusion relationships, and the noticeable separation between bodies and background. In this work, we define video body-swapping as an independent task and propose three critical consistencies: identity consistency, motion consistency, and environment consistency. We introduce an end-to-end model named SwapAnyone, treating video body-swapping as a video inpainting task with reference fidelity and motion control. To improve the ability to maintain environmental harmony, particularly luminance harmony in the resulting video, we introduce a novel EnvHarmony strategy for training our model progressively. Additionally, we provide a dataset named HumanAction-32K covering various videos about human actions. Extensive experiments demonstrate that our method achieves State-Of-The-Art (SOTA) performance among open-source methods while approaching or surpassing closed-source models across multiple dimensions. All code, model weights, and the HumanAction-32K dataset will be open-sourced at https://github.com/PKU-YuanGroup/SwapAnyone.

cs.CV↗

arXiv2025

OSCAR: Object Status and Contextual Awareness for Recipes to Support Non-Visual Cooking

Following recipes while cooking is an important but difficult task for visually impaired individuals. We developed OSCAR (Object Status Context Awareness for Recipes), a novel approach that provides recipe progress tracking and context-aware feedback on the completion of cooking tasks through tracking object statuses. OSCAR leverages both Large-Language Models (LLMs) and Vision-Language Models (VLMs) to manipulate recipe steps, extract object status information, align visual frames with object status, and provide cooking progress tracking log. We evaluated OSCAR's recipe following functionality using 173 YouTube cooking videos and 12 real-world non-visual cooking videos to demonstrate OSCAR's capability to track cooking steps and provide contextual guidance. Our results highlight the effectiveness of using object status to improve performance compared to baseline by over 20% across different VLMs, and we present factors that impact prediction performance. Furthermore, we contribute a dataset of real-world non-visual cooking videos with step annotations as an evaluation benchmark.

cs.HC↗

arXiv2025

Plasmon-enhanced Direct Detection Method for Boosted sub-MeV Dark Matter

Plasmon, a collective mode of electronic excitation in solid-state detectors, provides a novel way to detect light dark matter (DM). In this work, we present the conditions of DM to produce plasmon resonance, requiring relativistic velocities for light DM, and generalize the collective excitation framework to account for relativistic DM. As a demonstration, we consider the cosmic ray boosted DM (CRDM) and find that the plasmon resonance can be significantly enhanced in the scenario with a light mediator. Utilizing the first data from SENSEI experiment with the skipper-CCDs at SNOLAB, we obtain a new strong limit on the sub-MeV DM-electron scattering cross section.

hep-ph↗

arXiv2025

CookingDiffusion: Cooking Procedural Image Generation with Stable Diffusion

Recent advancements in text-to-image generation models have excelled in creating diverse and realistic images. This success extends to food imagery, where various conditional inputs like cooking styles, ingredients, and recipes are utilized. However, a yet-unexplored challenge is generating a sequence of procedural images based on cooking steps from a recipe. This could enhance the cooking experience with visual guidance and possibly lead to an intelligent cooking simulation system. To fill this gap, we introduce a novel task called \textbf{cooking procedural image generation}. This task is inherently demanding, as it strives to create photo-realistic images that align with cooking steps while preserving sequential consistency. To collectively tackle these challenges, we present \textbf{CookingDiffusion}, a novel approach that leverages Stable Diffusion and three innovative Memory Nets to model procedural prompts. These prompts encompass text prompts (representing cooking steps), image prompts (corresponding to cooking images), and multi-modal prompts (mixing cooking steps and images), ensuring the consistent generation of cooking procedural images. To validate the effectiveness of our approach, we preprocess the YouCookII dataset, establishing a new benchmark. Our experimental results demonstrate that our model excels at generating high-quality cooking procedural images with remarkable consistency across sequential cooking steps, as measured by both the FID and the proposed Average Procedure Consistency metrics. Furthermore, CookingDiffusion demonstrates the ability to manipulate ingredients and cooking methods in a recipe. We will make our code, models, and dataset publicly accessible.

cs.CV↗

arXiv2025

$\ell_0$ Factor Analysis: A P-Stationary Point Theory

Factor Analysis is a widely used modeling technique for stationary time series which achieves dimensionality reduction by revealing a hidden low-rank plus sparse structure of the covariance matrix. Such an idea of parsimonious modeling has also been important in the field of systems and control. In this article, a nonconvex nonsmooth optimization problem involving the $\ell_0$ norm is constructed in order to achieve the low-rank and sparse additive decomposition of the sample covariance matrix. We establish the existence of an optimal solution and characterize these solutions via the concept of proximal stationary points. Furthermore, an ADMM algorithm is designed to solve the $\ell_0$ optimization problem, and a subsequence convergence result is proved under reasonable assumptions. Finally, numerical experiments demonstrate the effectiveness of our method in comparison with some alternatives in the literature.

math.OC↗

arXiv2025

Benchmarking and Defending Against Indirect Prompt Injection Attacks on Large Language Models

The integration of large language models with external content has enabled applications such as Microsoft Copilot but also introduced vulnerabilities to indirect prompt injection attacks. In these attacks, malicious instructions embedded within external content can manipulate LLM outputs, causing deviations from user expectations. To address this critical yet under-explored issue, we introduce the first benchmark for indirect prompt injection attacks, named BIPIA, to assess the risk of such vulnerabilities. Using BIPIA, we evaluate existing LLMs and find them universally vulnerable. Our analysis identifies two key factors contributing to their success: LLMs' inability to distinguish between informational context and actionable instructions, and their lack of awareness in avoiding the execution of instructions within external content. Based on these findings, we propose two novel defense mechanisms-boundary awareness and explicit reminder-to address these vulnerabilities in both black-box and white-box settings. Extensive experiments demonstrate that our black-box defense provides substantial mitigation, while our white-box defense reduces the attack success rate to near-zero levels, all while preserving the output quality of LLMs. We hope this work inspires further research into securing LLM applications and fostering their safe and reliable use.

cs.CL↗

arXiv2025

Probing Supernova Neutrino Boosted Dark Matter with Collective Excitation

We explore the supernova neutrino-boosted dark matter (SN$ν$BDM) and its direct detection. During core-collapse supernovae, an abundance of neutrinos are emitted. These supernova neutrinos can transfer their kinetic energy to the light dark matter via their interactions, and thus are detected in the neutrino and dark matter experiments. Due to the exponential suppression of the high-energy neutrino flux, the kinetic energy carried by a large portion of SN$ν$BDM falls within the MeV range. This could potentially produce the collective excitation signals in the semiconductor detectors with the skipper-CCD. We calculate the plasmon excitation rate induced by SN$ν$BDM and derive the exclusion limits. In contrast with conventional neutrino and dark matter direct detection experiments, our results present a significant enhancement in sensitivity for the sub-MeV dark matter.

hep-ph↗