Search arXiv⌕ Search

arXiv subjects

Jonathan Ullrich

Publications and source records attributed to Jonathan Ullrich.

2 recordsLinked to original sources

Adapt the Object to the Subject: Representative Study Material for Empirical Software Engineering

External validity is a central concern in empirical software engineering, yet the debate has largely focused on whether study participants are representative. How well the study objects they reason about - code, models, or requirements - are representative of each participant's own work is often overlooked. Practitioners' own artifacts would be ideal, but are often out of reach, so researchers sample from public datasets or construct objects artificially. Whether this lack of subject-object fit affects participants' ability to act on the material remains unknown. We take a first step toward answering this question for requirements. We propose an approach that uses an LLM to generate context-adapted synthetic requirements just-in-time during an online survey from a short, participant-supplied context description, optionally injecting requirements smells as controlled stimuli. To validate the approach, we conducted a controlled experiment with 31 industry practitioners in a 2x2 design that crosses stimulus generation (static vs. synthetic) with quality (clean vs. smelly). Each participant rated the perceived representativeness and assessability of 20 requirements. Using Bayesian data analysis within a statistical causal-inference framework, we find that perceived representativeness has a strong positive effect on assessability. Synthetic requirements are rated as more representative and, primarily due to this gain, more assessable than static ones sourced from public datasets; smell injection leaves both perceptions unchanged. Our results support treating a lack of study-object representativeness as a first-class validity threat and position context-adapted synthetic study objects as a practical addition to the empirical methodology toolbox.

cs.SE↗

From Requirements to Code: Understanding Developer Practices in LLM-Assisted Software Engineering

With the advent of generative LLMs and their advanced code generation capabilities, some people already envision the end of traditional software engineering, as LLMs may be able to produce high-quality code based solely on the requirements a domain expert feeds into the system. The feasibility of this vision can be assessed by understanding how developers currently incorporate requirements when using LLMs for code generation-a topic that remains largely unexplored. We interviewed 18 practitioners from 14 companies to understand how they (re)use information from requirements and other design artifacts to feed LLMs when generating code. Based on our findings, we propose a theory that explains the processes developers employ and the artifacts they rely on. Our theory suggests that requirements, as typically documented, are too abstract for direct input into LLMs. Instead, they must first be manually decomposed into programming tasks, which are then enriched with design decisions and architectural constraints before being used in prompts. Our study highlights that fundamental RE work is still necessary when LLMs are used to generate code. Our theory is important for contextualizing scientific approaches to automating requirements-centric SE tasks.

cs.SE↗