arXiv subjects

Christoph Treude

Publications and source records attributed to Christoph Treude.

At least 19 recordsLinked to original sources

arXiv2026

Disposable Implementations: What Is Software Engineering For?

Existing implementations are usually maintained for most of their life and replaced only as an exception. Coding agents may change that. If agents can produce new implementations from requirements, tests, architectural constraints, and operational records that survive replacement, changing a system may no longer require modifying its existing codebase. We call this process regeneration. An implementation is disposable when regeneration becomes a routine alternative to maintenance. Regenerating and verifying a clean implementation from current requirements and tests may be cheaper than having agents navigate (and potentially compound) existing technical debt, and could eliminate much of that debt. Yet existing code may contain knowledge absent from the preserved artifacts, including safeguards introduced after failures and assumptions that were never documented. A regenerated implementation may pass the available tests and still lose behavior that matters in production. This leaves open how to establish continuity across replacements and whether regeneration reduces maintenance effort or shifts it to the artifacts and processes that support it. We examine these questions through an exploratory review of practitioner literature. Published examples show that agents can replace existing implementations, but do not establish that repeated regeneration is safe or economical. If implementations become disposable, what should software engineering preserve?

cs.SE↗

arXiv2026

AI Slop and the Software Commons

We argue that AI slop in software development is creating a tragedy of the commons. Slop is cheap to generate but expensive to review, and its costs fall on shared resources such as reviewer capacity, which was limited even before AI assistants became widespread. Ostrom documented how communities avoid such tragedies by setting and enforcing their own rules. We translate Ostrom's design principles for sustainable commons institutions into concrete proposals for contributors, reviewers, tool developers, team leads, organizational leaders, and educators.

cs.SE↗

arXiv2026

Ethics by Configuration: AI Ethics Principles in Coding Agent Instructions

As AI coding agents become integral to software development, projects increasingly utilise configuration files, such as context files and skills, to govern agent behaviour. While AI ethics principles are widely used in research and policy, little is known about whether and how they are reflected when developers configure coding agents. We studied ethics principles in 40,079 configuration files across 8,893 GitHub repositories. Using a validated large language model (LLM) classification, we measured the prevalence of ethics-related statements, and qualitatively analysed 702 stratified samples across 679 files to uncover the ethical considerations reflected in these files and the parties they concern. Ethics-related statements appeared in 39.4% of files and 61.5% of repositories; concentrated on accountability, transparency, privacy protection, and reliability; and occurred more often in task-specific files such as skills than in project-wide context files. Our qualitative analysis identified 30 categories of ethical considerations, which take the form of concrete rules for the agent. Accountability appeared mainly as requirements for human approval, transparency as explain and honestly report the agent's work, and fairness as accessibility and localisation. These considerations reveal a difference in focus. Whereas public ethics frameworks aim to protect society and end users, configuration files mainly concern developers, maintainers, and community members in the development workflow. Ethics-related content appeared mainly in prose-based files, and whether agents follow these instructions remains untested. We discuss the implications for practitioners, researchers, and policymakers.

cs.SE↗

arXiv2026

What Will Replace Pull Requests? Rethinking Software Collaboration for AI Agents

The emergence of artificial intelligence (AI) coding agents has put pull requests under pressure: agents open millions of pull requests per month, while some open-source projects have begun to restrict or disable external contributions. We investigate which alternatives to pull requests practitioners are proposing or adopting and what these changes mean for empirical software engineering research. We reviewed 50 gray-literature documents from 31 organizations or individuals published between October 2025 and October 2026, coding the problems each named, the mechanisms it proposed or adopted, which of five roles of pull requests (integration, review, rationale, accountability, and external contribution) each mechanism takes over, and the evidence offered. Most responses retained pull requests in some form, whether by adding tooling, attaching a new artifact, or restricting access, and restriction was the response most often reported as adopted. Several proposed alternatives moved review and rationale into agent conversations and session records, but only Zed reported using a replacement in practice. Only two documents outlined a replacement for external contribution, and neither reported adoption. Pull requests are changing in ways that pull-request datasets will not capture: reviews conducted in agent conversations leave no pull-request comments, and projects that disable pull requests disappear from samples of newly opened ones. We outline a research agenda on the effectiveness of the emerging alternatives, their consequences for open-source participation, and the adaptation of empirical methods.

cs.SE↗

arXiv2026

How Do Coding-Agent Harnesses Self-Evolve? An Empirical Study of Retrospective Adaptation

Coding agents can adapt their harnesses, the software, configuration, and instructions used to run them, based on experience from earlier tasks, but whether these adaptations improve their behavior on later tasks remains unclear. We study \retro, a skill that asks an agent to review a completed session and propose harness changes, and ask what changes it proposes and whether implemented changes reduce repeated work on later tasks. We ran Claude Code on 30 SWE-bench Verified issues, qualitatively coded the 125 proposals from \retro, implemented 24, and evaluated 12 harness changes on later issues from the same repositories in fresh sessions with and without each change. Most proposals referred to an event in the session, but 18 of 43 lint-related proposals recommended a practice regardless of what had happened. Among the seven evaluations in which sessions without the change encountered the targeted problem, two changes eliminated the targeted action, one reduced its occurrence, four showed no reduction, and some changes added work on unrelated issues. Harness self-evolution requires more than generating and implementing changes: it needs ways to establish whether a change helps on later tasks and to revise or remove changes that do not.

cs.SE↗

arXiv2026

Toward Evidence-Based Agentic Software Engineering

Task success rates and execution costs of agentic coding tools depend on both model and harness. The harness manages context, provides tools, and runs the agent loop. In empirical software engineering (SE) studies, comparisons that change model and harness together cannot isolate their effects. We qualitatively annotated 325 experiments reported in 146 peer-reviewed full papers published between 2025 and 2026. We examined the reported configurations and whether comparisons isolated harness effects, including harness configuration and version, model, tasks, and user interaction. The exact version of every harness used was reported for only 15 experiments. For five, the paper reported when the experiments were run, and for 31, authors shared execution traces for every session. Of the 226 experiments comparing harnesses, 35 never varied the harness in isolation. Based on this analysis, we derive requirements for a reference harness that the SE community can use for experiments. We call on the community to report all degrees of freedom of experiments with agentic coding tools and on SE venues to pilot dedicated tracks or special issues around a specified reference harness release. In these tracks, papers would hold the harness version fixed and vary features through separately versioned extensions to isolate feature effects and test their interactions.

cs.SE↗

arXiv2026

Developer Task Delegation to LLMs and the Trust Factor: A Mixed Methods Study

With the rise of Large Language Models (LLMs), software developers are presented with a powerful new approach: delegating software development tasks to LLMs. Existing studies have explored task delegation for a subset of tasks and factors. Critically, what tasks developers delegate to LLMs, to what extent, and why they delegate them, is not yet well understood. We explore these questions through 22 interviews and a survey with 160 software practitioners. Developers reported employing four modes of task delegation to LLMs: full delegation, partial delegation, self-delegation, and task avoidance. Drawing on our qualitative findings, we discuss the rationales behind these choices and explore how task difficulty, task risk, developer motivation, and trust appear to be associated with software development task delegation to LLMs. Our statistical analysis reveals that trust is a significant factor associated with task delegation to LLMs. Finally, we share actionable implications for human-AI collaboration in software research, and for software practitioners to make the most of their newly found powers of delegation.

cs.SE↗

arXiv2026

Loop Engineering: Building Blocks, Adoption, and Impact

Over the past months, the way developers guide agentic AI coding tools has moved up several levels of abstraction, from phrasing prompts to engineering context to configuring the harness around the underlying model. In June 2026, practitioners began to describe a new level called loop engineering: Instead of prompting an agent interactively, developers design systems that prompt agents for them. The term spread rapidly, accompanied by bold claims and vocal skepticism, but its adoption in software projects has not yet been measured. We present (1) an exploratory review of the emerging gray literature on loop engineering, (2) a traceability analysis and an exploratory mining study of 36,710 software repositories, and (3) the design of a controlled experiment to study how loop goals, static analysis checks, and decision policies affect task success, code quality, resource usage, and token consumption of agent loops. Our literature review found broad agreement on what a well-engineered loop contains: agent loops started on a schedule or by repository events and bounded by machine-checkable stop conditions, persistent state files, verifier subagents, token budgets, and defined points of escalation to humans. These sources provide no systematic evidence of the claimed benefits. In the mining study, we found autonomous agent loops in 217 open-source GitHub repositories using heuristics and behavioral signals. Nearly all of these repositories committed the configuration that starts their loops, but almost none of them committed the state files that the gray literature suggests. Our proposed experiment is a first step toward closing the evidence gap identified in our literature review. We call for further empirical studies of the benefits and costs of loop engineering for software engineering tasks.

cs.SE↗

arXiv2026

Acquiring and Verifying Repository Norms for Coding Agents

Changes produced by coding agents can pass functional tests while leaving repository contribution requirements unmet. Following repository-specific norms requires identifying guidance dispersed across repository sources and interpreting its conditions and exceptions. Retrieval and documentation approaches supply general context, but agents must still determine which norms apply. We introduce RepoNorm to acquire explicit and implicit repository norms independently of coding tasks. It checks norm content and applicability using repository evidence, consults Git history when needed, and delivers norm packages to existing coding agents. Our evaluation uses three coding models and 121 tasks from RepoNormBench. Against the baseline with no additional generated guidance (Raw), relative improvements are 7.42-10.77% for Overall Norm Compliance Rate (NCR), 31.64-45.44% for Contribution NCR, and 11.34-17.69% for Prompt-omitted NCR. All three coding models also obtain higher values on these NCR measures with RepoNorm than with CodeWiki documentation. Functional success rates show observed gains of 5.79-9.09 percentage points over Raw; the paired comparisons do not reach the significance threshold. Sampled precision is 86% with RepoNorm's default configuration.

cs.SE↗

arXiv2026

Enhancing the Non-Functional Quality Compliance of LLM-Generated Code through Quality-Aware Preference Learning

Large Language Models (LLMs) have been widely adopted in commercial code completion engines, significantly enhancing coding efficiency and productivity. However, even functionally correct LLM-generated code may exhibit non-functional quality issues that violate coding standards and best practices, such as poor style and limited maintainability. To address this, we propose a framework for quality-aware preference learning that guides LLMs toward generating criteria-compliant code. Our approach consists of three phases. First, we construct a dataset of paired criteria-violating and criteria-compliant samples, where each pair contains code exhibiting a specific non-functional quality issue and its repaired version that resolves the issue. Second, we design an adaptive token weighting mechanism to emphasize quality-sensitive code regions. Third, we introduce a hybrid optimization objective that combines ranking loss with language modeling loss and KL divergence to enable effective comparative optimization. Extensive experiments on DeepSeek-Coder and Qwen2.5-Coder show that our method substantially improves compliance with the targeted non-functional quality criteria while maintaining functional correctness, achieving a 75.7% relative increase in Quality Reciprocal Score (QRS) on MBPP-sanitized for Qwen2.5-Coder. Fine-tuning a 7B model requires less than three hours, indicating strong practical viability. Ablation studies and a user study further support the effectiveness of the proposed framework.

cs.SE↗

arXiv2026

Guidelines for Empirical Studies in Software Engineering involving Large Language Models

Large Language Models (LLMs) are widely used in software engineering (SE) research and practice, yet their non-determinism, opaque training data, and rapidly evolving models threaten the reproducibility and replicability of empirical studies. We address this challenge through a collaborative effort of 22 researchers, presenting a taxonomy of seven study types that organizes how LLMs are used in SE research, together with eight guidelines for designing and reporting such studies. Each guideline distinguishes requirements (must) from recommendations (should) and is contextualized by the study types it applies to. Our guidelines recommend that researchers: (1) declare LLM usage and role; (2) report model versions, configurations, and customizations; (3) document the system and prompt design beyond the model; (4) report session traces, i.e., interaction logs and runtime traces; (5) use suitable baselines, benchmarks, and metrics; (6) include an open LLM as a baseline; (7) validate LLM outputs against human judgment; and (8) articulate limitations and mitigations. We complement the guidelines with an applicability matrix mapping guidelines to study types and a reporting checklist for authors and reviewers. We maintain the study types and guidelines online as a living resource for the community to use and shape (llm-guidelines$.$org).

cs.SE↗

arXiv2026

Fine-grained Approaches for Confidence Calibration of LLMs in Automated Code Revision

In today's AI-assisted software engineering landscape, developers increasingly depend on LLMs that are highly capable, yet inherently imperfect. The tendency of these models to produce incorrect outputs can reduce developer productivity. To this end, a canonical mitigation method is to provide calibrated confidence scores that faithfully reflect their likelihood of correctness at the instance-level. Such information allows users to make immediate decisions regarding output acceptance, abstain error-prone outputs, and better align their expectations with the model's capabilities. Since post-trained LLMs do not inherently produce well-calibrated confidence scores, researchers have developed post-hoc calibration methods, with global Platt-scaling of sequence-level confidence scores proving effective in many generative software engineering tasks but remaining unreliable or unexplored for automated code revision (ACR) tasks such as program repair, vulnerability repair, and code refinement. We hypothesise that the coarse-grained nature of this conventional method makes it ill-suited for ACR tasks, where correctness is often determined by local edit decisions and miscalibration can be sample-dependent, thereby motivating fine-grained confidence calibration. To address this, our study proposes local Platt-scaling applied separately to three different fine-grained confidence scores. Through experiments across 3 separate tasks and correctness metrics, as well as 14 different models of various sizes, we find that fine-grained confidence scores consistently achieve lower calibration error across a broader range of probability intervals, and this effect is further amplified when global Platt-scaling is applied. Our proposed approaches offer a practical solution to eliciting well-calibrated confidence scores, enabling more trustworthy and streamlined usage of imperfect models in ACR tasks.

cs.SE↗

arXiv2026

AI Policies: Help or Hindrance? A Software Developer's Perspective

AI policies introduced by software organisations to mitigate LLM-related risks such as sensitive information leaks and unauthorised usage are not useful if software developers do not engage with them. We draw on 19 software developer interviews to show how AI policies help and hinder developers. We suggest approaches to support managers and decision makers with a developer-centric approach to introducing AI policies.

cs.SE↗

arXiv2026

Integrating Crash Report Mining and LLMs for Bug Localization and Repair: An Industrial Report

Analyzing crash-report bugs in large-scale industrial software systems requires substantial maintenance effort, particularly in production environments where developers must handle large volumes of crash reports and source code artifacts to localize and fix their root causes. While recent studies have shown that Large Language Models (LLMs) can assist with maintenance tasks, little is known about their effectiveness in supporting developers in analyzing crash-report bugs and repairing bugs associated with groups of crash reports in industrial settings. To address this gap, we investigate whether integrating crash report mining techniques---specifically stack trace clustering and suspicious file and method ranking---with LLMs can support crash localization and repair in production environments. We conduct a retrospective evaluation of five LLMs under four prompt configurations. After that, we chose the best model to run on 38 crash bugs collected from two large Java enterprise systems. We further analyze the structural characteristics and explanatory patterns of LLM-generated responses and assess localization and repair effectiveness through manual validation. Our results show that the best-performing configuration localizes up to 71% and correctly repairs 52% of crash bugs on the full dataset. These findings provide empirical evidence that combining crash report mining with LLM-based repair can effectively support debugging activities in industrial maintenance workflows.

cs.SE↗

arXiv2026

"An Endless Stream of AI Slop": How Developers Discuss the Burden of AI-Assisted Software Development

"AI slop", that is, low-quality AI-generated content, is increasingly affecting software development, from generated code and pull requests to documentation and bug reports. However, there is limited empirical research on how developers perceive and respond to this phenomenon. We qualitatively analyzed how developers discuss AI slop in 1,154 Reddit and Hacker News posts, developing a codebook of 15 codes organized into three thematic clusters: Review Friction (how AI slop burdens reviewers, erodes trust, and prompts countermeasures), Quality Degradation (damage to codebases, knowledge resources, and developer competence), and Forces and Consequences (systemic incentives, mandated adoption, craft erosion, and workforce disruption). Our findings frame AI slop as a tragedy of the commons, where individual productivity gains externalize costs onto reviewers, maintainers, and the broader community. We report the concerns developers raise and the mitigation strategies they propose, with implications for tool developers, team leads, and educators.

cs.SE↗

arXiv2026

Does My README File Need To Be Updated? Exploring LLM-Based README Maintenance

README files are critical for understanding and onboarding contributors to open-source software, yet they frequently become outdated. We formulate surgical documentation update recommendation as a task and present a Large Language Model-driven framework for use in a human-in-the-loop workflow. Given a pull request, the framework determines whether a README update is needed, identifies where changes should be made, and explains the triggering events. We evaluate the framework on 25,511 pull requests from 714 popular repositories. Its best configuration recovers half of the pull requests historically accompanied by README updates and achieves 28% user-facing accuracy under the observed prevalence of such updates. A qualitative failure analysis further identifies opportunities for improvement. We also conduct a retrospective study of 20 sampled repositories and a case study with a developer from a large open-source project. Manual annotation shows that 21.5% of temporally matched recommendations identify updates overlooked by developers, or 6.1% under the most conservative interpretation. These results suggest that the reported user-facing accuracy is a lower bound on likely deployment performance. Finally, we discuss implications for integrating documentation update tools into open-source development workflows.

cs.SE↗

arXiv2026

An Exploratory Study of Agent Plans for Agentic AI Coding Tools in Open-Source Software

Repository-level configuration artifacts allow developers to provide guidance for agentic AI coding tools, such as Claude Code, Gemini, etc. Although prior research has examined repository-shared context files that capture project-level instructions and conventions (e.g., AGENTS.md files), little is known about more task-oriented artifacts such as Agent Plans. We present an exploratory study of Agent Plans in open-source software repositories, examining how plan files are preserved, which development activities they support, and what information they provide to guide agent execution. We screened 36,710 GitHub repositories belonging to engineered software projects and identified 85 Markdown plan files from 10 repositories. Within this highly concentrated corpus, Agent Plans supported several kinds of software engineering work, including maintenance, design, construction, quality-related work, and process support. They also provided task-oriented execution guidance, most commonly through implementation steps, concrete files and locations, and testing and validation information. Overall, repository-preserved Agent Plans under these tool-specific directories appear to be a narrow but informative artifact for studying task intent and execution guidance in human-agent workflows.

cs.SE↗

arXiv2026

Automated Code Review Assignments: An Alternative Perspective of Code Ownership on GitHub

Code ownership is central to ensuring accountability and maintaining quality in large-scale software development. Yet, as external threats such as software supply chain attacks on project health and quality assurance increase, mechanisms for assigning and enforcing responsibility have become increasingly critical. In 2017, GitHub introduced the CODEOWNERS feature, which automatically designates reviewers for specific files to strengthen accountability and protect critical parts of the codebase. Despite its potential, little is known about how CODEOWNERS is actually adopted and practiced. We present the first large-scale empirical study of CODEOWNERS usage across over 844,000 pull requests with 1.9 million comments and over 2 million reviews. We identify 10,287 code owners to track their review activities. Results indicate that codeowners tend to adhere to the rules specified in the CODEOWNERS file, exhibit similar collaborative behaviors to traditional metrics of ownership, but tend to contribute to a smoother and faster PR workflow over time. Finally, using regression discontinuity design (RDD) analysis, we find that repositories adopting CODEOWNERS experience shifts in review dynamics, as ownership redistributes review responsibilities away from core developers. Our results position CODEOWNERS as a promising yet sparsely adopted mechanism for improving software governance and resilience. We discuss how projects can leverage this alternative ownership method as a perspective to enhance security, accountability, and workflow efficiency in open-source development.

cs.SE↗