Search arXiv⌕ Search
Home/Archive/arXiv

arXiv · 2309.12924

Automated grading workflows for providing personalized feedback to open-ended data science assignments

Federica Zoe Ricci·Catalina Mari Medina·Mine Dogucu
View original source ↗Read source document ↗Export metadata ↓

Abstract

Open-ended assignments - such as lab reports and semester-long projects - provide data science and statistics students with opportunities for developing communication, critical thinking, and creativity skills. However, providing grades and formative feedback to open-ended assignments can be very time consuming and difficult to do consistently across students. In this paper, we discuss the steps of a typical grading workflow and highlight which steps can be automated in an approach that we call automated grading workflow. We illustrate how gradetools, a new R package, implements this approach within RStudio to facilitate efficient and consistent grading while providing individualized feedback. By outlining the motivations behind the development of this package and the considerations underlying its design, we hope this article will provide data science and statistics educators with ideas for improving their grading workflows, possibly developing new grading tools or considering use gradetools as their grading workflow assistant.

Explore related subjects

stat.OTcs.CYphysics.ed-ph

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Federica Zoe Ricci, Catalina Mari Medina, Mine Dogucu. 2024-02-29. Automated grading workflows for providing personalized feedback to open-ended data science assignments. https://doi.org/10.5070/t5.1886

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Why is Regularization Underused? An Empirical Study on Trust and Adoption of Statistical Methods

Statistical practice does not automatically follow methodological innovation. Regularization methods, widely advocated to reduce overfitting and stabilize inference, are readily available in modern software, but are not consistently used by data analysts. We investigate this implementation gap in a large-scale empirical study of trust in, and acceptance of, regularization techniques, based on $N = 606$ data analysts. Drawing on measurement frameworks from technology acceptance research, we survey practitioners and embed a randomized experiment to test whether written recommendation of regularization methods increases trust or intended use. We find no evidence of such an effect. Instead, adoption intentions are strongly associated with analysts' perceptions of ease of implementation and practical benefit, such as improved bias control or interpretability. Perceived social norms also emerge as a central driver. These results indicate that uptake of statistical methodology depends less on formal recommendations than on usability, perceived utility, and community practice.

stat.OT↗
arXiv2026

Exact analysis of a split--merge queue with latent Erlang-factor dependent subtask times

This paper studies a two-server split--merge queue with positively dependent subtask service times modeled through a latent-factor bivariate Erlang construction. An exact characterization of the split--merge completion time is obtained, including explicit formulas for its first two moments and the resulting mean waiting time. Under fixed marginal service-time distributions, independence is shown to stochastically increase the completion time and hence overestimate mean waiting time. Numerical illustrations show that this benchmark gap can be substantial.

stat.OT↗
arXiv2026

Statistical Compatibility, Refutational Information, and Acceptability

This paper develops an interpretive framework for divergence P-values and S-values within a descriptive frequentist perspective. Statistical analysis is framed as operating within idealized worlds defined by a set of assumptions and a target hypothesis, where probabilities describe the behavior of data under the model but do not assign truth values to hypotheses. Within this view, P-values are interpreted as graded indices of compatibility between the observed result and the predictions generated by the assumed model; accordingly, small P-values should not be read as indicating logical impossibility or strict inconsistency of the model itself. Building on this distinction, the paper argues that practical inference requires moving beyond the internal logic of the model toward judgments of overall acceptability, which depend not only on data-model compatibility but also on multiple contextual considerations such as subject-matter knowledge, plausibility of assumptions, data quality, usefulness, and loss - all interpreted through the competence, intentions, perceptions, and moral values of the specific analyst. S-values are therefore interpreted not as evidence against the epistemic status of the model, but as a specific form of refutational information that contributes to the broader body of information used by the analyst to judge whether a model remains acceptable for an intended practical purpose. The paper also examines the linguistic and conceptual risks associated with the language of incompatibility, distinguishes probability from rarity, and clarifies different notions of surprise - including a possible definition of Shannon-type surprise, to be distinguished from Bayesian belief revision. Overall, the article proposes a more cautious and explicit interpretation of frequentist measures, centered on model-based description, analyst responsibility, and decision acceptability.

stat.OT↗