Search arXiv⌕ Search

arXiv subjects

Saman Rahbar

Publications and source records attributed to Saman Rahbar.

6 recordsLinked to original sources

Nanopath: Fast, Fair, Open Experimentation for Pathology Foundation Models

We introduce nanopath, an open-source framework and benchmark for rapidly training and evaluating small-scale pathology foundation models. Using a single H100 GPU, a typical nanopath model trains in a little over an hour on up to one million TCGA pathology tile presentations and evaluates within 25 minutes on a broad suite of tile-level and slide-level tasks. Nanopath enables rapid exploration of research ideas at small scale, so promising training recipes can subsequently be tested with more data and compute. To enable fair comparisons between ideas, we constrain submissions to fixed data and compute budgets while keeping the downstream evaluation protocol fixed. Anyone may clone the codebase, automatically download the data, reproduce or modify existing models, and submit a run to Labless, our public experiment ledger where researchers and AI agents can search through prior attempts. Since its public release in June 2026, nanopath has received 1,269 experiments from 20 contributors. Here, we detail its design and assess how well its rapid evaluation tracks standardized THUNDER, HEST, and PathoBench (CPTAC) benchmarks across six nanopath and 14 reference foundation models. Nanopath's fast evaluation suite strongly tracked these standardized benchmarks, and a few 22M parameter nanopath models even outperformed giant 1.1B parameter models on a subset of downstream tasks. We subsequently discuss how we have since optimized nanopath's evaluation suite to be an even more faithful, quick proxy for external benchmarking. Nanopath provides an accessible testbed for students, researchers, and AI agents to explore pathology foundation modeling through fair, transparent, and directly comparable experiments.

cs.CV↗

The Score Is Not the Structure: Brain Alignment and Cross-Lingual Transfer

Similarity scores are often offered as evidence that a model shares structure with the brain or across languages. We ask what such a score reads when that structure is removed, or when the instrument measuring it does not work, in two settings. Across seventeen languages, a grammaticality probe transfers worse between more distant languages (r = -0.66). But the probe's own accuracy falls along the same axis and is at chance in four languages, four of the five most distant. Dropping them halves the explained variance, and because it also narrows the range of distances, the design cannot say how much of the gradient is the instrument. Counting the 272 language pairs as independent gives p = 0.0006 for a steering effect that is null when the seventeen languages are the unit (p = 0.155). In brain alignment, a training objective raises a language model's similarity to fMRI responses from 0.10 to 0.34 (reliability ceiling 0.54), yet targets with the brain correspondence destroyed still reach 0.31. A shuffled target scores about k/n against the real one, for rank k and n sentences, a baseline that can be computed before any model is trained. Both interventions work, yet we detect no distance-graded steering effect and no syntactic benefit from alignment. Before reading a correspondence score, measure how much of it survives without the correspondence.

cs.CL↗

Site Is Decodable Before Pretraining: Negative Controls for Probing Frozen Brain-MRI Foundation Models

Brain foundation models are often tested with a probe. The model is frozen, a simple classifier is trained on its output, and the classifier's accuracy is taken as a sign of what pretraining learned. We show that this reading can be wrong unless two controls are reported with it. We probed three frozen 3-D brain-MRI models at five depths, on two independent multi-site cohorts (ABIDE-I and ABIDE-II). The site where a scan was acquired could be predicted at about 0.9, on a scale where 0 is chance and 1 is perfect. No clinical or demographic target came close. An untrained copy of the same model predicted site just as well on both cohorts. Untrained models of three architectures, each built with three random seeds, did the same on ABIDE-I. The raw image, shrunk to 12x12x12 voxels and given to no model at all, predicted site at 0.95. Pretraining did not create the site signal. It was already in the scans. We also looked for any clinical target on which pretraining beats an untrained model, and found no reliable case. Adjusting for age, sex and diagnosis left site prediction almost unchanged, and so did a 4.6-fold increase in the number of voxels. Resting-state fMRI from the same people showed the same pattern. Site was the most predictable attribute of brain connectivity (0.81), and it survived five untrained layers while age did not. We recommend reporting two controls with every probe of a brain foundation model: running the probe again on the untrained model, and on the raw input. We release the code for both.

cs.CV↗

The Curse of Multilinguality in Lexical Normalization

Lexical normalization rewrites the noisy, non-standard words that fill user-generated text (tmrw, u, gr8) into their standard forms. Because labelled data is scarce for most languages, a popular shortcut is to train a single model on many languages at once. We ask a simple question: how many languages should such a model be trained on? Using one fixed-capacity character-level model and twelve languages from a standard benchmark, we vary the number of jointly trained languages from one to twelve and measure per-language accuracy. We find a clear curse of multilinguality: accuracy is highest when a language is trained with only a few others, often just one to four, and then falls steadily and substantially, dropping by about forty percent as the rest are piled on. A control that holds the total amount of training data constant makes the decline arrive sooner and fall further, which points to competition among the languages for one fixed-size model rather than to how much data is available. We also test whether a language's typological distance from the others predicts its ideal number of co-training languages, and find no dependable rule: any apparent relationship rests on a couple of languages and does not hold up. For compact normalization models, less can be more: a few languages beat pooling everything into a single model.

cs.CL↗

Topic Matching in the Wild: Benchmark and Lessons from Real-World ASR Transcripts

In contact centers, real-time agent-assist tools determine, for each of many predefined topics, whether a live customer utterance is relevant and display a coaching card to the agent when it is. The input is noisy and challenging: ASR(Automatic Speech Recognition) transcripts of spontaneous phone conversations, which can be unclear, repetitive, and mostly lack punctuation. To systematically study this real-world task, we curate a human-annotated topic-utterance judgments dataset sourced from real call-center transcripts. We compare three types of matchers: a regex-based baseline, zero-shot sentence embedding encoders, and Gemini-based LLM matchers. In addition, two types of topic representations are studied in our benchmark:keyphrases and natural language description. Our empirical experiments highlight the superior performance of lightweight LLM matchers over embedding and regex models when equipped with natural language descriptions.

cs.CL↗

The Ignition Index: Measuring Global Workspace Dynamics in Language Models

We introduce the Ignition Index (I), a validated scalar metric that operationalizes Global Workspace Theory's (GWT) all-or-none ignition prediction in transformer language models. The metric fits a four-parameter sigmoid to per-layer linear probe accuracy as a function of input signal strength, extracting steepness parameter beta-hat: high values indicate abrupt, ignition-like transitions; low values indicate graded build-up. Across 11 models spanning five architecture families, shuffled-label controls demonstrate 9.6-fold selectivity for genuine linguistic structure over spurious probe capacity (p < 0.001, Mann-Whitney U-test). We find: (1) Feedforward transformers exceed SSMs by 89% in aggregate beta-hat (p < 1e-13, Cohen's d = 0.52), with Mamba exhibiting near-linear profiles consistent with absent global broadcast. (2) Huginn-3.5B exhibits 2.12-fold higher ignition along its iteration axis than its depth axis, demonstrating that recurrent architectures manifest workspace-like transitions along the recurrence dimension. (3) Pythia-410M shows a PELT-detected phase transition at training step 256 (+67%), preceding induction-head formation. (4) Hypotheses linking ignition to model scale and signal strength were not confirmed, suggesting transformer architectures may saturate available ignition mechanisms. The Ignition Index provides the first validated quantitative bridge between GWT's dynamical predictions and mechanistic interpretability, with 9.6-fold measurement selectivity and architecture-level discriminability not previously characterized in the scaling literature. Code: https://github.com/saman-rahbar/ignition-index

cs.AI↗