AegisBench: Auditing Aerial Search-and-Rescue Person Detection Under the Conditions It Is Deployed In

Abstract

Drone-based person detection is being adopted by emergency-management agencies to find missing people faster than ground teams can. The benchmarks behind that adoption were collected in calm, clear daylight, while the missions that trigger deployment happen in floods, wildfires, storms, and at night. AegisBench measures that gap. It applies nine physically motivated corruptions from four disaster families, each at three severities monotone in a declared image statistic or physical quantity, to two aerial search-and-rescue datasets, HERIDAL and SARD. Three architecturally distinct detectors (Faster R-CNN, YOLOv11, RT-DETR) are scored across 168 conditions by one shared evaluator, at confidence thresholds frozen on clean validation data as a fielded system would run. Robustness is highly uneven. Low light is the extreme case: a 4.5-stop exposure loss drives recall to exactly zero for all three detectors on both datasets, a result that holds under threshold-free mAP and across three training seeds, and is mirrored on 300 real night drone images, where 2 of 759 people are found. Surviving detections largely stay on target, so the failure is silent: an empty frame looks the same as one whose occupants the system cannot see. Fine-tuning on milder low-light levels recovers 0.06 to 0.19 recall at the held-out severity, at an architecture-dependent cost on other conditions, and does not transfer to real night imagery. Spatial corruptions rank differently on the two datasets, which we cannot attribute to altitude alone because their parameters scale with frame size rather than person size. We release the corruption engine, evaluation pipeline, and all per-condition results as an audit instrument for anyone evaluating these systems.