arXiv subjects

Gun Ahn

Publications and source records attributed to Gun Ahn.

3 recordsLinked to original sources

arXiv2026

Closed-Loop Agent Evaluation: Layered Measurement and Self-Calibration

Once an agent is live, teams want to know whether it is doing its job, not just whether it throws errors. Yet most evaluation is one-off: a benchmark run, a look at traces, a single score. Its conclusions neither check themselves nor reach the agent. We propose closed-loop evaluation, an evaluation method for deployed agents. It models evaluation as performance estimation under biased raters and censored outcomes, and it requires every level's conclusion to trace back to atomic data and to be computed by versioned rules. The method covers four levels of analysis (span, trace, conversation and engagement), measures along a spine of Execution, Substance, Trajectory and Results, treats Conduct as a constraint throughout, and uses Provenance and Adjudication components to certify the evidence and the evaluators. Evaluator agents generated by an Agent Factory carry these components, each with a domain knowledge pack and an adapter harness. Two loops close the evaluation: an inner loop audits the evidence, rules and metrics themselves, and an outer loop uses per-component regression checks to drive agent improvement. We also give a pipeline that runs this process automatically. Using the business-analysis agent SharpInsight as a case and porting the method to public conversation data, we show how problems differ across levels and how evidence-and-rule judgment is more reliable than holistic scoring (ICC 0.86 vs. 0.81; kappa 0.81 vs. 0.63).

cs.AI↗

arXiv2026

Beyond the Flag: Clinical Framing Closes the Moderation Gap in Suicide Risk Measurement

Moderation APIs are built to flag policy-violating content, not to measure graded clinical risk. But a platform's duty does not end at detection: the response owed to passive distress differs sharply from the response owed to active planning with means access, and emerging regulation (e.g., California Senate Bill 243) is turning that distinction into a compliance requirement. We therefore ask how well deployed safety signals recover clinically meaningful severity. We release a benchmark of 516 r/SuicideWatch posts rated by a licensed psychiatrist on a four-level ordinal schema (Indicator, Ideation, Behavior, Attempt) grounded in the Columbia Suicide Severity Rating Scale, and evaluate moderation APIs, prompted LLMs, and supervised baselines under seven ordinal-aware metrics. Three findings. Vendor moderation APIs separate low- from high-severity posts well (0.860 high-risk F1) but measure severity poorly (0.395 macro F1), systematically over-predicting the most severe category. Clinically grounded zero-shot prompting recovers much of that gap (0.562 macro F1), and expert-authored framing (not fine-tuning, added reasoning, or naive multi-agent aggregation) is the effective lever. The value of reasoning depends on register: it hurts on long, noisy Reddit posts and helps on short, clinician-authored statements. We argue graded severity, not a binary flag, is what a proportionate duty of care requires, and release our evaluation framework to support that measurement.

cs.CL↗

arXiv2019

Measurement Anomaly of Step Width in Calibration Grating using Atomic Force Microscopy

We imaged the topography of a silicon grating with atomic force microscopy (AFM) using different scan parameters to probe the effect of pixel pitch on resolution. We found variations in the measured step height and profile of the grating depending on scan parameters, with measured step width decreasing from 1300 to 108 nm and step height increasing from 172 to 184 nm when a pixel pitch in the scan axis decreased from 625 nm to 3.91 nm. In order to resolve the measurement anomaly of step width, we compared these values with step width and height of the same grating measured using scanning electron microscopy (SEM). The values obtained from SEM imaging were 187.3 nm +/- 6.2 nm and 116 nm +/- 10.4 nm, which were in good agreement with AFM data using a 3.91 nm of pixel pitch. We think that we need at least four pixels over the step width to avoid the measurement anomaly induced by the stick-slip or dragging of the tip. Our findings that RMS roughness varied less than 1 nm and converged at the value of 77.6 nm for any pixel pitch suggest that the RMS roughness is relatively insensitive to the pixel pitch.

physics.app-ph↗