Search arXiv⌕ Search
Home/Archive/arXiv

arXiv · 2407.09050

Refusing Safe Prompts for Multi-modal Large Language Models

Zedian Shao·Hongbin Liu·Yuepeng Hu·Neil Zhenqiang Gong
View original source ↗Read source document ↗Export metadata ↓

Abstract

Multimodal large language models (MLLMs) have become the cornerstone of today's generative AI ecosystem, sparking intense competition among tech giants and startups. In particular, an MLLM generates a text response given a prompt consisting of an image and a question. While state-of-the-art MLLMs use safety filters and alignment techniques to refuse unsafe prompts, in this work, we introduce MLLM-Refusal, the first method that induces refusals for safe prompts. In particular, our MLLM-Refusal optimizes a nearly-imperceptible refusal perturbation and adds it to an image, causing target MLLMs to likely refuse a safe prompt containing the perturbed image and a safe question. Specifically, we formulate MLLM-Refusal as a constrained optimization problem and propose an algorithm to solve it. Our method offers competitive advantages for MLLM model providers by potentially disrupting user experiences of competing MLLMs, since competing MLLM's users will receive unexpected refusals when they unwittingly use these perturbed images in their prompts. We evaluate MLLM-Refusal on four MLLMs across four datasets, demonstrating its effectiveness in causing competing MLLMs to refuse safe prompts while not affecting non-competing MLLMs. Furthermore, we explore three potential countermeasures-adding Gaussian noise, DiffPure, and adversarial training. Our results show that though they can mitigate MLLM-Refusal's effectiveness, they also sacrifice the accuracy and/or efficiency of the competing MLLM. The code is available at https://github.com/Sadcardation/MLLM-Refusal.

Explore related subjects

cs.CRcs.AIcs.CVcs.LG

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Zedian Shao, Hongbin Liu, Yuepeng Hu, Neil Zhenqiang Gong. 2024-09-05. Refusing Safe Prompts for Multi-modal Large Language Models. https://arxiv.org/abs/2407.09050

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Studying Detection Rule Generation as a Unified Task

Security systems use detection rules to identify suspicious activity. Existing studies often investigate rule generation for specific security systems, devoting substantial effort to developing dedicated methods and evaluation setups. Such customization contributes to fragmented research, limiting method reuse and result comparability across systems. We therefore study detection rule generation as a unified task across diverse natural language inputs and rule languages. To support method reuse, we propose UniRule, which abstracts diverse rules into shared natural language representations for retrieval. To enable consistent evaluation, we introduce a protocol that compares rules under shared criteria and aggregates the results into method scores. Experiments demonstrate the effectiveness of UniRule and the reliability of the evaluation protocol. They also show that method performance in one setting can be predicted from results in others, with average error close to that obtained using that setting's own data. These findings support studying detection rule generation as a unified task.

cs.CR↗
arXiv2026

0%, 45%, or 99%: A Guardrail's Own Share of the Refusals It Is Credited With

A defended pipeline's refusals have two producers: the guardrail bolted in front of the model, and the model's own alignment. Recovering the split costs nothing, because a guard block replaces the model's response and the two counts are therefore disjoint. Holding the defense, the targets, the corpus and the judge fixed, the guardrail's own share of the refusals credited to it is 0%, 41-45%, or 99% across three settings that a results table would describe identically. Two choices move it, and neither belongs to the deployer who bought the guardrail. The attacker drives the share to zero by choosing which channel carries the payload: a plainly written request rendered as pixels, with nothing obfuscated, leaves a text guard's read covering none of it. The evaluator drives the share to 99% by choosing what text fills a defense's internal slots: fill them with the unencoded request behind an encoded attack, a read no deployed defender possesses, and the same guard blocks almost everything. The two consequences differ, and only the attacker's can happen to a running system. The evaluator's choice is an artifact carried by the literature, and its size is set by where the granted text lands: substantial at a guard gate, smaller in a caption-mediated re-check, absent in a majority-vote smoother, an ordering reproduced in an independent replicate. Isolating the grant inside the caption-mediated defense refutes the prediction we registered, since the harm-verdict stage contributes nothing while the stage that regenerates the answer carries the whole effect. The reference implementation builds every stage from a single prompt field that cannot represent the difference between what the attacker sent and what the benchmark records, and an audit of four further released harnesses and of the benchmark itself finds the same structural gap, so faithful porting supplies the grant silently.

cs.CR↗
arXiv2026

Reflex-Guard: A Low-Latency Guardrail for LLM Prompt Safety Using Dense Semantic Embeddings

Large Language Models (LLMs) in real-world applications often face the risks of specially crafted prompts designed to bypass the safety controls. Existing guardrail methods, such as LLM-as-a-judge and cloud-based safety APIs are able to detect unsafe content. However, they often add a delay of about 250-900 ms to each request. This delay is too high for real-time applications, when the system usually needs to respond in less than 100 ms. Furthermore, routing user prompts through external moderation endpoints raises significant data privacy concerns. This paper introduces Reflex-Guard, a lightweight guardrail that runs locally. It uses jailbreak-aware preprocessing, compact sentence-transformer embeddings, and seven fast binary classifiers. Together, these components enable high-accuracy prompt safety filtering with much lower latency than existing solutions. Through systematic evaluation on a strategically balanced dataset of 30,568 samples drawn from five complementary sources, we demonstrate that Reflex-Guard achieves 95.9% recall on harmful prompts at 37.6 ms end-to-end latency. It is faster than existing baselines, including Llama Guard 2 at 255 ms and SafeDecoding at 723 ms. It can detect 100% of GCG suffix attacks and Base64-encoded prompts using the default threshold. However, DrAttack structured prompts required lowering the threshold to 0.03 for optimal detection, as they produced a distinct probability distribution. Reflex-Guard achieves Reflex Efficiency Score (RES) scores up to 16.79, significantly outperforming Llama Guard 2 (11.90) and SafeDecoding (9.80). This analysis offers practical deployment advice and shows that different attack types occupy distinct regions in the embedding probability space.

cs.CR↗