Search arXiv⌕ Search
Home/Archive/arXiv

arXiv · 2610.08097

When Tools Lie: Reliability of Mathematical Agents Under Corrupted Tool Feedback

Kavienan Jegatheesan·Gayathri Lihinikaduarachchi
View original source ↗Read source document ↗Export metadata ↓

Abstract

Mathematical problem solving often requires deterministic computational steps that agents delegate to tools and implicitly trust. Yet tools can fail silently, returning plausible but incorrect results. How well can agents detect and correct corrupted tool call outputs? We study this through a controlled corruption framework where a hidden interceptor replaces tool call results with plausible incorrect information on targeted problems. We evaluate agents across 31 problems under four verification designs including no verification (baseline), mandatory same-context reflection, optional fresh-context verification, and optional structural verification. Without verification, corruption causes dramatic accuracy loss, from 100% down to 72.4%. Mandatory reflection fully recovers this performance to 100%. Optional verification improves accuracy only when models actively invoke it. Our results show that checking frequency is strongly associated with robustness differences, while unequal invocation prevents a controlled comparison of verifier quality. A supporting recovery experiment shows that full problem restart succeeds in 100% of cases after explicit detection. These findings demonstrate that verifier availability and verification policy are separate components of mathematical-agent reliability. Mandatory policies enforce verification while optional policies depend on the model's own choice to invoke it.

Explore related subjects

cs.CRcs.AIcs.SE

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Kavienan Jegatheesan, Gayathri Lihinikaduarachchi. 2026-10-06. When Tools Lie: Reliability of Mathematical Agents Under Corrupted Tool Feedback. https://arxiv.org/abs/2610.08097

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Adversarial Images Hijack Web Agents from Visual Grounding to Browser Execution

Modern web agents built on large vision-language models process webpages, select relevant UI elements, and translate model outputs into browser actions. Existing visual red-teaming approaches use adversarial visual content to manipulate this process. However, they primarily target model inference and do not explicitly account for structured input processing or action post-processing. Consequently, model-level success does not establish control over browser execution and cannot reliably characterize end-to-end agent robustness. To address this gap, we formulate red teaming for vision-grounded web agents as an end-to-end grounding-to-execution problem, and introduce WebMirage, a framework that crafts localized visual perturbations that cause agents to select attacker-controlled content and execute the corresponding browser action across varying webpage renderings. It uses a role-slot abstraction and webpage recomposition to capture competition among webpage elements, and dataflow analysis to align optimization with action post-processing. We evaluate WebMirage across four agent configurations and six VLM backbones on 2,250 tasks covering 13 public websites and a sandbox benchmark. WebMirage achieves an average attack success rate of 91.9%, compared with 17.4% for the strongest baseline, and remains effective against three agent-level defenses.

cs.CR↗
arXiv2026

ICS-Sniper: A Targeted Blackhole Attack on Encrypted ICS Traffic

Modern industrial control systems (ICS) increasingly host their Supervisory Control and Data Acquisition (SCADA) services in the cloud to reduce the costs of large-scale automation. To protect site-SCADA communications, ICS operators commonly use VPN tunneling and standard security practices. We show that, despite these security measures, an on-path Internet adversary can disrupt ICS operations without infiltrating the ICS perimeter, breaking encryption, or knowledge of the control logic. We present ICS-Sniper, a targeted blackhole attack that analyzes the VPN traffic metadata (sizes, direction, timing of packets) to identify narrow time windows, called critical superperiods, during which the site-SCADA traffic would likely contain highly critical commands or data. Post-analysis, in a subsequent operational cycle, ICS-Sniper drops a small set of payload-carrying packets in the critical superperiods to disrupt the ICS's operations. We demonstrate three attacks on two realistic modern Secure Water Treatment (SWaT) plant testbeds that can potentially violate the operational safety of the ICS while evading state-of-the-art ICS attack detectors.

cs.CR↗
arXiv2026

Large Language Models for Cryptocurrency Transaction Analysis: A Bitcoin Case Study

Cryptocurrencies are widely used, yet current methods for analyzing transactions often rely on opaque, black-box models. While these models may achieve high performance, their outputs are usually difficult to interpret and adapt, making it challenging to capture nuanced behavioral patterns. Large language models (LLMs) have the potential to address these gaps, but their capabilities in this area remain largely unexplored, particularly in cybercrime detection. In this paper, we test this hypothesis by applying LLMs to real-world cryptocurrency transaction graphs, with a focus on Bitcoin, one of the most studied and widely adopted blockchain networks. We introduce a three-tiered framework to assess LLM capabilities: foundational metrics, characteristic overview, and contextual interpretation. This includes a new, human-readable graph representation format, LLM4TG, and a connectivity-enhanced transaction graph sampling algorithm, CETraS. Together, they significantly reduce token requirements, transforming the analysis of multiple moderately large-scale transaction graphs with LLMs from nearly impossible to feasible under strict token limits. Experimental results demonstrate that LLMs have outstanding performance on foundational metrics and characteristic overview, where the accuracy of recognizing most basic information at the node level exceeds 98.50% and the proportion of obtaining meaningful characteristics reaches 95.00%. Regarding contextual interpretation, LLMs also demonstrate strong performance in classification tasks, even with very limited labeled data, where top-3 accuracy reaches 72.43% with explanations. While the explanations are not always fully accurate, they highlight the strong potential of LLMs in this domain. At the same time, several limitations persist, which we discuss along with directions for future research.

cs.CR↗