arXiv subjects

Aritz Bilbao-Jayo

Publications and source records attributed to Aritz Bilbao-Jayo.

3 recordsLinked to original sources

arXiv2026

Would an LLM Match into a Residency? Uncertainty, Reasoning and Image Use of Open Language Models on the Spanish MIR Examination

Accuracy on medical examinations is common evidence offered for the clinical use of language models, yet a health system that intends to run an open model on its own hardware needs to know considerably more: whether the model recognizes when it is likely to be wrong, whether medical fine-tuning improves this ability, whether its correct answers rest on sound clinical reasoning, or whether it actually uses the images it is given. We examine these questions for four open models of 8 to 27 billion parameters on the Spanish MIR residency examination, scoring 1,552 questions from eight editions under the exam's own rules. Answering every question, the three larger models would rank among roughly the first 600 to 3,000 candidates, with 41 to 43 specialties still open to them. Abstention, however, never pays, because even their least confident answers are right more often than the 25% break-even. Medical fine-tuning reduced overconfidence at the cost of accuracy, without improving how well confidence separates right from wrong answers. Moreover, 13-38% of correct answers rejected a distractor for a clinically unsound reason that confidence did not reflect, and removing or swapping the provided contextual images left accuracy unchanged.

cs.CL↗

arXiv2026

A Counterfactual Graph Approach to Oppositional Narrative Analysis

Distinguishing legitimate critical speech from coordinated conspiracy narratives is a pressing challenge for the integrity of the online information ecosystem, yet prevailing approaches rely on labeled entity roles and rigid predefined schemas that fail to capture the diffuse and context-dependent nature of participation in such narratives. We propose a modular framework that extracts an entity-interaction graph directly from narratives and reasons over it for classification, without depending on annotated node-level labels. Our pipeline enumerates candidate entity spans from a domain-adapted BERT encoder and learns a bipartite graph structure, which is then classified with a Heterogeneous Graph Transformer. To make predictions interpretable, we cast the graph as a hypergraph and apply a counterfactual distillation procedure that estimates the Individual Treatment Effect of each node, pruning extraneous entities to recover a minimal predictive subgraph. On the standard PAN Oppositional Narratives benchmark, our model achieves a macro F1-score of 0.93 and an MCC of 0.84, reaching state-of-the-art performance with roughly one-third of the parameters of the next best system, while pruning the graphs by 75.2% on average without altering predictions. Our results demonstrate that graph-based representations coupled with counterfactual inference offer both strong predictive accuracy and transparency into the entities driving conspiratorial discourse.

cs.CL↗

arXiv2026

When Does Gene Regulatory Network Inference Break? A Controlled Diagnostic Study of Causal and Correlational Methods on Single-Cell Data

Despite theoretical advantages, causal methods for Gene Regulatory Network (GRN) inference from single-cell RNA-seq data consistently fail to match or outperform correlation-based baselines in many realistic benchmarks, a persistent puzzle which casts doubt on the value of causality for this task. We argue that existing benchmarks are insufficiently controlled to answer this question because they evaluate on real or semi-real data where multiple pathologies co-occur, confounding failure modes, and obscuring the specific conditions under which different inference methods excel or fail. To address this gap, we introduce a controlled diagnostic framework that isolates seven biologically motivated pathologies (dropout, latent confounders, cell-type mixing, feedback loops, network density, sample size, and pseudotime drift) and measure how six representative methods spanning three inference paradigms degrade as each pathology intensifies. Across 6,120 controlled experiments, we find that causal methods genuinely dominate in clean and structurally favorable regimes, but specific pathologies (notably dropout and latent confounders) selectively neutralize their advantages. We further introduce an error-type decomposition that reveals methods with similar aggregate accuracy commit qualitatively different errors. To probe whether single-pathology effects persist when multiple stressors co-occur, we perform an interaction sweep over the three most impactful pathologies and find that their joint effects are sub-additive, while also exposing density-conditional cross-overs invisible to single-dial analysis. Our findings offer a nuanced understanding of when and why different methods succeed or fail for GRN inference, providing actionable insights for method development and practical guidance for practitioners.

cs.LG↗