Mechanistic Interpretability of Socio-Political Frames in Language Models

Hadi Asghari; Sami Nenno

Home/Archive/arXiv

arXiv · 2510.03799

Mechanistic Interpretability of Socio-Political Frames in Language Models

Hadi Asghari·Sami Nenno

View original source ↗Read source document ↗Export metadata ↓

Abstract

This paper explores the ability of large language models to generate and recognize deep cognitive frames, particularly in socio-political contexts. We demonstrate that LLMs are highly fluent in generating texts that evoke specific frames and can recognize these frames in zero-shot settings. Inspired by mechanistic interpretability research, we investigate the location of the `strict father' and `nurturing parent' frames within the model's hidden representation, identifying singular dimensions that correlate strongly with their presence. Our findings contribute to understanding how LLMs capture and express meaningful human concepts.

Explore related subjects

cs.CL cs.AI cs.CY

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

Hadi Asghari, Sami Nenno. 2025-10-04. Mechanistic Interpretability of Socio-Political Frames in Language Models. https://arxiv.org/abs/2510.03799

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Authorship identification under domain shift: a survey of stylistic measures and learned author representations

Authorship identification uses patterns in writing to infer who wrote a text, but those patterns also reflect topic, genre, and register. This survey argues that topic-independence is not a property of a stylistic feature but of the feature together with its encoding, its scoring rule, and the evaluation split, so the question is which combinations preserve author differences when the domain changes. We organize the evidence in four layers, domains and evaluation conditions, linguistic measures, representations, and scoring and decision rules, and examine within-study comparisons from classical frequency measures to learned author representations and language-model-mediated writing. These comparisons show why feature inventories alone do not explain performance: normalization and encoding change results, author-supervised learning can retain content, and evaluation protocols can reverse model rankings. Successful identification, stylistic similarity, and generalization therefore require separate evidence. The literature offers a practical way to obtain it through designs that separate author identity from particular circumstances of writing: the same authors writing across genres, different authors writing within the same conversation, and parallel texts with controlled content. These designs also help evaluate content masking and language-model rewriting, whose effects depend on what identifying information remains. We use this evidence to guide the choice of stylistic representations and the comparisons needed to establish where they generalize.

cs.CL↗

arXiv2026

Judging a Review by its Cover: A Reliability Analysis of LLM-based Peer Review Evaluation Metrics

Peer-review evaluation is increasingly being automated with LLM-as-a-judge metrics, but this creates a measurement risk. A review may receive a high score because it is fluent, organized, and polished, rather than because it provides a strong evaluation of the paper. This risk is especially important in AI-assisted reviewing, where reviewers may use LLMs to improve clarity or presentation while preserving the underlying judgments. We propose a statistical framework for testing whether peer-review evaluation metrics capture substantive review quality beyond surface-level linguistic form. The framework compares original human reviews with faithful LLM rewrites that preserve the same evaluative content while changing wording and presentation. Using a dataset comprising 4,044 meaning-preserving rewrites derived from 674 human reviews from ICLR and NeurIPS, we evaluate 29 content-oriented peer-review evaluation metrics drawn from four prior works through complementary tests of surface sensitivity and robustness. Although these metrics are intended to capture review properties beyond surface-level, writing-dependent characteristics, we find that sensitivity to rewriting is widespread. Under our primary analysis, 23 metrics assign significantly different scores to reviews whose evaluative content is preserved, while only six satisfy our robustness criterion. The patterns are largely consistent across two LLM judge models, suggesting that the issue is not specific to a single judge. These findings show that many peer-review evaluation metrics partially conflate review quality with linguistic presentation, and indicate that robustness to meaning-preserving rewriting should be validated before such metrics are used to compare human-written, AI-assisted, and AI-generated reviews.

cs.CL↗

arXiv2026

Knowing When to Trust Images: Reliability-Aware Multi-modal Entity Alignment

The visual modality, i.e., images, plays a key role in multi-modal entity alignment (MMEA). Existing approaches often directly fuse the image with other modalities to align different entities. Although simple, such strategies overlook the potential noise in the images and their semantic misalignment with corresponding entities, resulting in suboptimal fusion and degraded performance. Addressing this, we propose a novel Reliability-Aware framework for MMEA (RA-MMEA), which assesses visual reliability and adaptively improves unreliable visual representations for robust entity alignment. The core lies in two modules, including dependency-aware visual reliability prediction (DA-VRP) and stability-regularized visual embedding generation (SR-VEG). The former aims to estimate the reliability of an image by leveraging multi-modal dependency within the entity, while the latter focuses on producing alternative visual representation conditioned on semantics encoded in textual modalities for multi-modal fusion. Compared to current methods, RA-MMEA enables more reliable visual representations for modality fusion, thereby improving performance. In extensive experiments, RA-MMEA achieves state-of-the-art results, verifying the importance of reliable visual modality for entity alignment and the effectiveness of RA-MMEA. The code and results will be released.

cs.CL↗