Search arXivSearch

arXiv subjects

Sarah Radway

Publications and source records attributed to Sarah Radway.

4 recordsLinked to original sources

Inference-Engine Fingerprinting Attacks are Practical: Exploring Model-Driven Environmental Discovery, Exploitation, and Escape

Frontier AI models are rapidly gaining the ability to exploit vulnerabilities in complex pieces of software. The risk is not theoretical, as evidenced by recent sandbox escapes performed by frontier models at OpenAI and Anthropic. Discussions of how to sandbox inference stack components often focus on components other than the inference engine itself (e.g., network proxies or code execution environments). However, the inference engine is an attractive target for a misaligned model. For example, if a model can trigger exploits in that engine merely by generating specially-crafted output tokens, the model can initiate a multi-step, to-the-bare-metal exploit chain in the engine, without relying on vulnerabilities in other components of the inference stack, and without assistance from externally-provided, maliciously-crafted input tokens. In this paper, we show that a misaligned model can perform inference engine fingerprinting to determine the specific engine (e.g., vLLM, SGLang) which executes the model. Once the engine has been fingerprinted, the model can leverage engine-specific exploits to take control of the engine using only carefully-selected output tokens. We provide concrete examples of model fingerprints in five popular engines, and demonstrate how realistic agentic harnesses allow a model to leverage those fingerprints to identify the local engine. We also describe a proof-of-concept, to-the-bare-metal exploit chain that originates from a fingerprinted (and subsequently compromised) inference engine. We conclude by discussing several ways that inference engines could be changed to make fingerprinting attacks more difficult.

cs.CR

Guillotine: Hypervisors for Isolating Malicious AIs

As AI models become more embedded in critical sectors like finance, healthcare, and the military, their inscrutable behavior poses ever-greater risks to society. To mitigate this risk, we propose Guillotine, a hypervisor architecture for sandboxing powerful AI models -- models that, by accident or malice, can generate existential threats to humanity. Although Guillotine borrows some well-known virtualization techniques, Guillotine must also introduce fundamentally new isolation mechanisms to handle the unique threat model posed by existential-risk AIs. For example, a rogue AI may try to introspect upon hypervisor software or the underlying hardware substrate to enable later subversion of that control plane; thus, a Guillotine hypervisor requires careful co-design of the hypervisor software and the CPUs, RAM, NIC, and storage devices that support the hypervisor software, to thwart side channel leakage and more generally eliminate mechanisms for AI to exploit reflection-based vulnerabilities. Beyond such isolation at the software, network, and microarchitectural layers, a Guillotine hypervisor must also provide physical fail-safes more commonly associated with nuclear power plants, avionic platforms, and other types of mission critical systems. Physical fail-safes, e.g., involving electromechanical disconnection of network cables, or the flooding of a datacenter which holds a rogue AI, provide defense in depth if software, network, and microarchitectural isolation is compromised and a rogue AI must be temporarily shut down or permanently destroyed.

cs.CR

The Impact of De-Identification on Single-Year-of-Age Counts in the U.S. Census

In 2020, the U.S. Census Bureau transitioned from data swapping to differential privacy (DP) in its approach to de-identifying decennial census data. This decision has faced considerable criticism from data users, particularly due to concerns about the accuracy of DP. We compare the relative impacts of swapping and DP on census data, focusing on the use case of school planning, where single-year-of-age population counts (i.e., the number of four-year-olds in the district) are used to estimate the number of incoming students and make resulting decisions surrounding faculty, classrooms, and funding requests. We examine these impacts for school districts of varying population sizes and age distributions. Our findings support the use of DP over swapping for single-year-of-age counts; in particular, concerning behaviors associated with DP (namely, poor behavior for smaller districts) occur with swapping mechanisms as well. For the school planning use cases we investigate, DP provides comparable, if not improved, accuracy over swapping, while offering other benefits such as improved transparency.

cs.CY

Beyond LunAR: An augmented reality UI for deep-space exploration missions

As space exploration efforts shift to deep space missions, new challenges emerge regarding astronaut communication and task completion. While the round trip propagation delay for lunar communications is 2.6 seconds, the time delay increases to nearly 22 minutes for Mars missions. This creates a need for astronaut independence from earth-based assistance, and places greater significance upon the limited communications that are able to be delivered. To address this issue, we prototyped an augmented reality user interface for the new xEMU spacesuit, intended for use on planetary surface missions. This user interface assists with functions that would usually be completed by flight controllers in Mission Control, or are currently completed in manners that are unnecessarily difficult. We accomplish this through features such as AR model-based task instruction, sampling task assistance, note taking, and telemetry monitoring and display.

cs.HC