arXiv · 2611.09846
Ensemble Methodology for Reference-Free Confidence Estimation in LLM-based Code Summarization
Abstract
This paper describes an approach and use case to estimate the correctness of LLM-generated COBOL summaries without access to human-authored references. The core intuition is that inter-model agreement on semantics is informative about truth, and that an LLM-as-judge can provide higher-fidelity pairwise assessments than traditional string- or embedding-based metrics. We operationalise this intuition with (i) controlled summary generation from diverse models; (ii) a LLM evaluator that compares pairs of summaries and emits structured judgments; and (iii) a calibrated aggregation rule that combines supportive evidence while penalising only corroborated contradictions.
Explore related subjects
Keep this discovery
Explore connections, maps & timelines
Marco Colombo, Balaji A Jayaram, Sowmyakumari Selvaraj, Sumit Tokkar, Animesh Kumar, Aravinda Prasad Sistla, Ravishankar Shetty, Rigel Gjomemo. 2026-11-06. Ensemble Methodology for Reference-Free Confidence Estimation in LLM-based Code Summarization. https://arxiv.org/abs/2611.09846
Cite the original work for its findings. Save a collection to share your selection of sources.