Edge-Inference Governors Need Memory-Clock State

Abstract

On integrated edge SoCs whose memory fabric is governed independently of the compute clocks, frequency-aware latency estimators let deadline-aware DVFS governors schedule ML inference by modeling latency over CPU and GPU clocks -- but they do not condition on the memory clock (EMC), a deployment state that decides whether a governor meets its deadlines and at what energy. We show this with a deployed, measured governor on Jetson Orin: an EMC-blind GPU-only fit misses 25-28% of cycles at tight deadlines, whereas an EMC-aware two-cell refit holds misses to <=0.9% under a 2% QoS budget -- selecting a budget-feasible operating point proactively, where latency-only reactive calibration cannot repair memory-clock-dependent slope error. Across six models on two Orin SKUs, the core MobileNetV2 and ViT-Small results replicate on both boards; detection and LLM deployments reproduce the failure on the NX. Sustained deployment requires two further state layers: decode horizon, where KV-cache growth erodes tight-deadline feasibility over long responses, and GPU co-tenancy, whose occupancy opens queueing tails. A contract-admission policy composes the three layers from bounded probes and deploys live, including a joint cell decoding 2,000 tokens against an active GPU co-tenant where tenancy-blind admission misses 32% of tokens. Under the reference-probe maximum-guard accounting, every accepted contract is decisively measured-feasible. Fresh-probe stress tests then expose a probe-variance failure mode at knife-edge admissions; a dispersion-banded guard, validated on held-out trials, eliminates the observed probe-induced low-clock selections (held-out aggregate 1.19%); and one bin of headroom yielded observed per-launch compliance across eight fresh launches. A governor's guarantee is thus a measured conservatism ladder -- state repair, probe-variance banding, and headroom for launch realization.