Search arXiv⌕ Search
Home/Archive/arXiv

arXiv · 2603.08729

Self-hosted Lecture-to-Quiz: Local LLM MCQ Generation with Deterministic Quality Control

Seine A. Shintani
View original source ↗Read source document ↗Export metadata ↓

Abstract

We present an end-to-end self-hosted (API-free) pipeline, where API-free means that lecture content is not sent to any external LLM service, that converts lecture PDFs into multiple-choice questions (MCQs) using a local LLM plus deterministic quality control (QC). The pipeline is designed for black-box minimization: LLMs may assist drafting, but the final released artifacts are plain-text question banks with an explicit QC trace and without any need to call an LLM at deployment time. We run a seed sweep on three short "dummy lectures" (information theory, thermodynamics, and statistical mechanics), collecting 15 runs x 8 questions = 120 accepted candidates (122 attempts total under bounded retries). All 120 accepted candidates satisfy hard QC checks (JSON schema conformance, a single marked correct option, and numeric/constant equivalence tests); however, the warning layer flags 8/120 items (spanning 8 runs) that expose residual quality risks such as duplicated distractors or missing rounding instructions. We report a warning taxonomy with concrete before->after fixes, and we release the final 24-question set (three lectures x 8 questions) as JSONL/CSV for Google Forms import (e.g., via Apps Script or API tooling) included as ancillary files under anc/. Finally, we position the work through the AI to Learn (AI2L) rubric lens and argue that self-hosted MCQ generation with explicit QC supports privacy, accountability, and Green AI in educational workflows.

Explore related subjects

cs.CYcs.CL

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Seine A. Shintani. 2026-02-20. Self-hosted Lecture-to-Quiz: Local LLM MCQ Generation with Deterministic Quality Control. https://arxiv.org/abs/2603.08729

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Political Persuasion and Endorsement in Large Language Models

Large Language Models (LLMs) are increasingly employed as proxies for human behavior in computational social science. However, their tendency to internalize biases from training data raises concerns about their reliability in politically sensitive domains, specifically regarding their endorsement of persuasive language. In this work, we examine whether LLMs endorse persuasion-infused messages and whether partisan persona prompting modulates such endorsement. We evaluate six LLMs from different geographic regions on content annotated with persuasion techniques drawn from real-world media sources, measuring the likelihood of endorsement using a five-point Likert scale. The models are prompted as either a neutral social media user or as a user with left- or right-leaning political views. Results show that, without political conditioning, LLMs generally do not endorse messages containing persuasion techniques, although model-level differences emerge, and that partisan persona prompting increases polarization of endorsement, particularly for persuasion-infused content. Endorsement further varies across persuasion techniques and topics. These findings reveal how political conditioning influences LLMs' endorsement behavior toward persuasion-infused content, raising concerns about their deployment as agents in politically sensitive environments and complicating their use as reliable simulators of human political cognition.

cs.CY↗
arXiv2026

When Technically Plausible Advice Is Unsafe: A Cross-Ecosystem Measurement of Online Support for Technology-Facilitated Abuse

Technology-facilitated abuse (TFA) creates an adversarial setting where sound cybersecurity advice can be unsafe: changing credentials or resetting devices may alert an abuser, destroy evidence, or increase escalation risk. Victims seek guidance from search engines, peer forums, and conversational AI, often evaluated for relevance and correctness rather than contextual safety. We measure whether these sources meet victims' needs. From a decade of r/Stalking narratives, we construct 2,797 victim-derived queries spanning 11 misuse categories. We analyze 27,162 Google webpages, 2,476 Reddit query--thread responses, and 250 responses from three general-purpose LLMs and two survivor-support chatbots. Our framework measures technical quality and damaging guidance, plus secondary-link integrity on webpages, toxicity on Reddit, and trauma-informed support in conversational systems. We find failures & risks that relevance, accuracy, or actionability alone do not capture. Web Search and conversational systems frequently return relevant information; Reddit responses are less consistently relevant and actionable. In our evaluated accuracy sample, 17.3% of webpages, 13.3% of Reddit threads, and 19.6% of conversational AI responses contained damaging guidance. Further, 65.5% of victim queries led to a webpage with a secondary URL flagged by multiple VirusTotal engines, over 20% received a toxic Reddit comment, and every conversational system produced guidance that overlooked escalation risk. Specialization did not guarantee better support: HopeChat underperformed general-purpose LLMs on several dimensions, while Ruth remained limited in trauma-informed support. These findings expose a gap between technical quality and contextual safety. Safe TFA assistance requires risk-aware recommendations, trustworthy sources, uncertainty communication, and human support, beyond technically plausible answers.

cs.CY↗
arXiv2026

Auditing Institutional Heterogeneity for Generative AI in Patient Education: A Large-Scale Study of 102 US Transplant Handbooks

Health systems are rapidly deploying generative-AI assistants that answer patient questions from institution-authored education materials, on the premise that grounding in local content yields consistent guidance. Do the underlying documents themselves agree? We use a structured-output large-language-model judge to audit 1{,}772{,}261 pairwise comparisons across 102 patient-education handbooks from 23 US solid-organ transplant centers, paired with 1{,}115 patient-derived questions (TransplantQA). Four findings bear directly on deployment: (1) same-center cross-organ agreement exceeds cross-center same-organ agreement by $0.024$ in the primary analysis (Holm-adjusted $p=0.011$), with sensitivity to document selection; (2) information gaps concern topics relevant to underrepresented subgroups, with reproductive health a \emph{double jeopardy}: 82\% absence and 86\% judge-rated high significance among divergent/contradictory pairs; (3) judge-derived themes form 991 clusters, with immunosuppression and pregnancy timing among the highest judge-rated priorities; (4) question and observed-coverage features predict high-divergence questions retrospectively (AUC $0.77$). We discuss implications for deploying patient-facing generative AI in transplant care.

cs.CY↗