Dynamic Extended Generalized Many-Facet Rasch Model: A Unified Approach to Rater-Mediated Assessment

Abstract

Rating procedures are crucial in psychological and medical diagnosis, educational assessment, peer review, and grant evaluation. A major methodological challenge is disentangling multiple sources of variability in ratings, such as subject ability, rater behavior, and item characteristics. Item Response Theory-based models, most notably many-facet Rasch models and hierarchical rater models, have been widely adopted for jointly estimating subject, rater, and item effects, while retaining high interpretability. We introduce the Dynamic Extended Generalized Many-Facet Rasch model, a general class of Bayesian semiparametric models that integrate and extend existing approaches. Its measurement model allows location and discrimination parameters to depend jointly on item and rater characteristics and incorporates subject-item and rater-subject variation. The dynamic extension introduces a rater-specific cross-lagged effect capturing dependence on the previously rated subject, and a Bayesian P-spline capturing systematic changes in discrimination over the rating sequence, accounting for learning, fatigue, or adaptation effects. Subject heterogeneity is addressed through Bayesian nonparametric mixtures, flexibly learning the latent ability distribution without strict parametric assumptions and enabling model-based subject clustering. We also derive a closed-form, model-based inter-rater consensus coefficient and study its characteristics. We discuss theoretical properties, develop posterior computation strategies, and evaluate the model in simulation studies and two real-data applications.