arXiv subjects

Marsha Chechik

Publications and source records attributed to Marsha Chechik.

At least 19 recordsLinked to original sources

arXiv2026

Property-Driven Synthetic Data Engineering for Data-Scarce Software Systems: Reflections from the Breast Cancer Domain

Modern software systems increasingly depend on data for analysis, prediction, testing, and decision-making. Yet many important domains, including medicine, safety-critical systems, and regulated industries, lack abundant, shareable, or representative data. Synthetic data generation is often proposed as a remedy, but our experience engineering software for intraoperative radiotherapy (IORT) in breast cancer treatment suggests that synthetic data shifts rather than solves the central engineering problem. The key challenge becomes deciding which properties synthetic data must preserve, how these properties should be elicited from stakeholders, how they can be validated under privacy constraints, and how they evolve. We call this problem property-driven synthetic data engineering. Drawing on a collaboration with oncologists and preliminary experiments with a sensitive IORT dataset, we identify challenges in requirements, validation, privacy, and pipeline evolution. We argue that automated software engineering research should develop methods and tools for eliciting, formalizing, checking, and evolving validity properties for synthetic data in data-scarce software systems.

cs.SE↗

arXiv2026

Compositional Generator Equivalence (Extended Version)

Property-based testing (PBT) is a powerful technique for software verification that relies on random input generators and "shrinking" processes to find and minimize counterexamples to executable specifications called properties. While optimizing these generators is crucial for testing efficiency, formally justifying such optimizations is currently difficult because existing languages lack a compositional semantics that is coarse-grained enough for high-level reasoning. In this paper, we first provide a formal account of the syntax and semantics of Hedgehog, a popular PBT framework. We demonstrate that Hedgehog's distribution semantics - which models how users typically reason about generators - is non-compositional. Furthermore, we prove that any sound and complete compositional semantics for Hedgehog must necessarily be equivalent to its sampling semantics, which is too fine-grained to justify common program optimizations. To resolve this dilemma, we introduce Hedgehog$^\rightarrow$, a restricted version of the language based on the arrow calculus, and prove that Hedgehog$^\rightarrow$ possesses a compositional distribution semantics. We evaluate Hedgehog$^\rightarrow$ through a Haskell implementation and show that it remains expressive enough to capture generators of practical interest, while providing the formal foundation needed for compositional generator equivalence proofs.

cs.PL↗

arXiv2026

Risk Under Pressure: Compute-Aware Evaluation of Adversarial Robustness in Language Models

Adversarial robustness evaluations of large language models (LLMs) typically report attack success rate (ASR) under fixed query budgets, implicitly treating all attacks as equally costly. In practice, the computational expense of different attack strategies can vary by orders of magnitude. Consequently, ASR at a fixed budget can obscure the true effort required to jailbreak a model, thereby making it hard to determine whether an attack's cost justifies its payoff to the attacker. We propose a compute-aware evaluation framework based on computational pressure, measured in cumulative floating-point operations (FLOPs), as a proxy for adversarial effort. We introduce risk-compute curves, which map compute budgets to attack risk, and derive two metrics that summarize the average pressure required for a given attack to succeed. Across ten models spanning three families and four different stages in language model training and alignment, evaluated with three attack strategies (gradient-based, iterative refinement, and template-based) on two jailbreak robustness benchmarks, we find: (1) alignment training has non-monotonic effects on compute-space robustness; (2) scaling model size reduces gradient-based attack effectiveness but has limited impact on cheaper template-based attacks; (3) gradient-based attacks optimized on a surrogate model can transfer to a separate target model, providing a way to reduce attacker costs; (4) compute cost varies by up to ${\approx}5{\times}$ across harm categories within a single model; and (5) safety-aligned RL increases aggregate cost while leaving some categories disproportionately accessible. We release our framework to enable compute-aware risk assessment and evaluation.

cs.LG↗

arXiv2026

Automated Repair of Requirements for Cyber-Physical Systems in Simulink Requirements Tables

The development of complex software systems, e.g., cyber-physical systems (CPSs), involves continuous evolution of both system implementations and their requirements. These two artifacts often proceed independently, creating a risk of misalignment. For example, a system may be updated due to implementation-level concerns, yielding a new version that no longer satisfies its original requirements. Traditional compliance recovery techniques, e.g., automated program repair, address this problem by modifying the system while assuming that requirements are correct. However, faulty, outdated or inadequate requirements are a well-documented challenge in practice, motivating the complementary task of requirement repair. In this paper, we propose a framework that leverages system execution data to repair misaligned CPS requirements, thereby restoring requirement-to-system compliance. Our approach evaluates the correctness of declarative requirements over time-based, real-valued signals expressed using the MATLAB Simulink Requirements Tables language. We evaluate seven variants of our framework on six real-world case studies covering 12 requirements. Results confirm the effectiveness of the proposed framework in producing correct and useful repaired requirements.

cs.SE↗

arXiv2026

Social, Legal, Ethical, Empathetic and Cultural Norm Operationalisation for AI Agents

As AI agents are increasingly used in high-stakes domains like healthcare and law enforcement, aligning their behaviour with social, legal, ethical, empathetic, and cultural (SLEEC) norms has become a critical engineering challenge. While international frameworks have established high-level normative principles for AI, a significant gap remains in translating these abstract principles into concrete, verifiable requirements. To address this gap, we propose a systematic SLEEC-norm operationalisation process for determining, validating, implementing, and verifying normative requirements. Furthermore, we survey the landscape of methods and tools supporting this process, and identify key remaining challenges and research avenues for addressing them. We thus establish a framework - and define a research and policy agenda - for developing AI agents that are not only functionally useful but also demonstrably aligned with human norms and values.

cs.AI↗

arXiv2026

Multi-CoLoR: Context-Aware Localization and Reasoning across Multi-Language Codebases

Large language models demonstrate strong capabilities in code generation but struggle to navigate complex, multi-language repositories to locate relevant code. Effective code localization requires understanding both organizational context (e.g., historical issue-fix patterns) and structural relationships within heterogeneous codebases. Existing methods either (i) focus narrowly on single-language benchmarks, (ii) retrieve code across languages via shallow textual similarity, or (iii) assume no prior context. We present Multi-CoLoR, a framework for Context-aware Localization and Reasoning across Multi-Language codebases, which integrates organizational knowledge retrieval with graph-based reasoning to traverse complex software ecosystems. Multi-CoLoR operates in two stages: (i) a similar issue context (SIC) module retrieves semantically and organizationally related historical issues to prune the search space, and (ii) a code graph traversal agent (an extended version of LocAgent, a state-of-the-art localization framework) performs structural reasoning within C++ and QML codebases. Evaluations on a real-world enterprise dataset show that incorporating SIC reduces the search space and improves localization accuracy, and graph-based reasoning generalizes effectively beyond Python-only repositories. Combined, Multi-CoLoR improves Acc@5 over both lexical and graph-based baselines while reducing tool calls on an AMD codebase.

cs.SE↗

arXiv2026

Operationalizing Human Values in the Requirements Engineering Process of Ethics-Aware Autonomous Systems

Operationalizing human values alongside functional and adaptation requirements remains challenging due to their ambiguous, pluralistic, and context-dependent nature. Explicit representations are needed to support the elicitation, analysis, and negotiation of value conflicts beyond traditional software engineering abstractions. In this work, we propose a requirements engineering approach for ethics-aware autonomous systems that captures human values as normative goals and aligns them with functional and adaptation goals. These goals are systematically operationalized into Social, Legal, Ethical, Empathetic, and Cultural (SLEEC) requirements, enabling automated well-formedness checking, conflict detection, and early design-time negotiation. We demonstrate the feasibility of the approach through a medical Body Sensor Network case study.

cs.SE↗

arXiv2026

Future of Software Engineering Research: The SIGSOFT Perspective

As software engineering conferences grow in size, rising costs and outdated formats are creating barriers to participation for many researchers. These barriers threaten the inclusivity and global diversity that have contributed to the success of the SE community. Based on survey data, we identify concrete actions the ACM Special Interest Group on Software Engineering (SIGSOFT) can take to address these challenges, including improving transparency around conference funding, experimenting with hybrid poster presentations, and expanding outreach to underrepresented regions. By implementing these changes, SIGSOFT can help ensure the software engineering community remains accessible and welcoming.

cs.SE↗

arXiv2026

Abductive Vibe Coding (Extended Abstract)

When software artifacts are generated by AI models ("vibe coding"), human engineers assume responsibility for validating them. Ideally, this validation would be done through the creation of a formal proof of correctness. However, this is infeasible for many real-world vibe coding scenarios, especially when requirements for the AI-generated artifacts resist formalization. This extended abstract describes ongoing work towards the extraction of analyzable, semi-formal rationales for the adequacy of vibe-coded artifacts. Rather than deciding correctness directly, our framework produces a set of conditions under which the generated code can be considered adequate. We describe current efforts towards implementing our framework and anticipated research opportunities.

cs.SE↗

arXiv2025

Assurance Case Development for Evolving Software Product Lines: A Formal Approach

In critical software engineering, structured assurance cases (ACs) are used to demonstrate how key system properties are supported by evidence (e.g., test results, proofs). Creating rigorous ACs is particularly challenging in the context of software product lines (SPLs), i.e, sets of software products with overlapping but distinct features and behaviours. Since SPLs can encompass very large numbers of products, developing a rigorous AC for each product individually is infeasible. Moreover, if the SPL evolves, e.g., by the modification or introduction of features, it can be infeasible to assess the impact of this change. Instead, the development and maintenance of ACs ought to be lifted such that a single AC can be developed for the entire SPL simultaneously, and be analyzed for regression in a variability-aware fashion. In this article, we describe a formal approach to lifted AC development and regression analysis. We formalize a language of variability-aware ACs for SPLs and study the lifting of template-based AC development. We also define a regression analysis to determine the effects of SPL evolutions on variability-aware ACs. We describe a model-based assurance management tool which implements these techniques, and illustrate our contributions by developing an AC for a product line of medical devices.

cs.SE↗

arXiv2025

Solving Set Constraints with Comprehensions and Bounded Quantifiers

Many real applications problems can be encoded easily as quantified formulas in SMT. However, this simplicity comes at the cost of difficulty during solving by SMT solvers. Different strategies and quantifier instantiation techniques have been developed to tackle this. However, SMT solvers still struggle with quantified formulas generated by some applications. In this paper, we discuss the use of set-bounded quantifiers, quantifiers whose variable ranges over a finite set. These quantifiers can be implemented using quantifier-free fragment of the theory of finite relations with a filter operator, a form of restricted comprehension, that constructs a subset from a finite set using a predicate. We show that this approach outperforms other quantification techniques in satisfiable problems generated by the SLEEC tool, and is very competitive on unsatisfiable benchmarks compared to LEGOS, a specialized solver for SLEEC. We also identify a decidable class of constraints with restricted applications of the filter operator, while showing that unrestricted applications lead to undecidability.

cs.LO↗

arXiv2025

Proceedings of the First International Workshop on Autonomous Systems Quality Assurance and Prediction with Digital Twins

This volume contains the proceedings of the First International Workshop on Autonomous Systems Quality Assurance and Prediction with Digital Twins (ASQAP 2025), which was held in Hamilton, Canada, on May 4th, 2025, as a satellite event of ETAPS 2025. The aim of ASQAP 2025 is to gather experts from academia and industry to explore the potential of digital twin technology in supporting quality assurance in autonomous systems, including concepts such as specification, verification, validation, testing, analysis, and many others.

cs.SE↗

arXiv2025

LEGOS-SLEEC: Tool for Formalizing and Analyzing Normative Requirements

Systems interacting with humans, such as assistive robots or chatbots, are increasingly integrated into our society. To prevent these systems from causing social, legal, ethical, empathetic, or cultural (SLEEC) harms, normative requirements specify the permissible range of their behaviors. These requirements encompass both functional and non-functional aspects and are defined with respect to time. Typically, these requirements are specified by stakeholders from a broad range of fields, such as lawyers, ethicists, or philosophers, who may lack technical expertise. Because such stakeholders often have different goals, responsibilities, and objectives, ensuring that these requirements are well-formed is crucial. SLEEC DSL, a domain-specific language resembling natural language, has been developed to formalize these requirements as SLEEC rules. In this paper, we present LEGOS-SLEEC, a tool designed to support interdisciplinary stakeholders in specifying normative requirements as SLEEC rules, and in analyzing and debugging their well-formedness. LEGOS-SLEEC is built using four previously published components, which have been shown to be effective and usable across nine case studies. Reflecting on this experience, we have significantly improved the user interface of LEGOS-SLEEC and its diagnostic support, and demonstrate the effectiveness of these improvements using four interdisciplinary stakeholders. Showcase video URL is: https://youtu.be/LLaBLGxSi8A

cs.CY↗

arXiv2024

Diagnosis via Proofs of Unsatisfiability for First-Order Logic with Relational Objects

Satisfiability-based automated reasoning is an approach that is being successfully used in software engineering to validate complex software, including for safety-critical systems. Such reasoning underlies many validation activities, from requirements analysis to design consistency to test coverage. While generally effective, the back-end constraint solvers are often complex and inevitably error-prone, which threatens the soundness of their application. Thus, such solvers need to be validated, which includes checking correctness and explaining (un)satisfiability results returned by them. In this work, we consider satisfiability analysis based on First-Order Logic with relational objects (FOL*) which has been shown to be effective for reasoning about time- and data-sensitive early system designs. We tackle the challenge of validating the correctness of FOL* unsatisfiability results and deriving diagnoses to explain the causes of the unsatisfiability. Inspired by the concept of proofs of UNSAT from SAT/SMT solvers, we define a proof format and proof rules to track the solvers' reasoning steps as sequences of derivations towards UNSAT. We also propose an algorithm to verify the correctness of FOL* proofs while filtering unnecessary derivations and develop a proof-based diagnosis to explain the cause of unsatisfiability. We implemented the proposed proof support on top of the state-of-the-art FOL* satisfiability checker to generate proofs of UNSAT and validated our approach by applying the proof-based diagnoses to explain the causes of well-formedness issues of normative requirements of software systems.

cs.SE↗

arXiv2024

Abstraction Engineering

Modern software-based systems operate under rapidly changing conditions and face ever-increasing uncertainty. In response, systems are increasingly adaptive and reliant on artificial-intelligence methods. In addition to the ubiquity of software with respect to users and application areas (e.g., transportation, smart grids, medicine, etc.), these high-impact software systems necessarily draw from many disciplines for foundational principles, domain expertise, and workflows. Recent progress with lowering the barrier to entry for coding has led to a broader community of developers, who are not necessarily software engineers. As such, the field of software engineering needs to adapt accordingly and offer new methods to systematically develop high-quality software systems by a broad range of experts and non-experts. This paper looks at these new challenges and proposes to address them through the lens of Abstraction. Abstraction is already used across many disciplines involved in software development -- from the time-honored classical deductive reasoning and formal modeling to the inductive reasoning employed by modern data science. The software engineering of the future requires Abstraction Engineering -- a systematic approach to abstraction across the inductive and deductive spaces. We discuss the foundations of Abstraction Engineering, identify key challenges, highlight the research questions that help address these challenges, and create a roadmap for future research.

cs.SE↗

arXiv2024

PLACIDUS: Engineering Product Lines of Rigorous Assurance Cases

In critical software engineering, structured assurance cases (ACs) are used to demonstrate how key properties (e.g., safety, security) are supported by evidence artifacts (e.g., test results, proofs). ACs can also be studied as formal objects in themselves, such that formal methods can be used to establish their correctness. Creating rigorous ACs is particularly challenging in the context of software product lines (SPLs), wherein a family of related software products is engineered simultaneously. Since creating individual ACs for each product is infeasible, AC development must be lifted to the level of product lines. In this work, we propose PLACIDUS, a methodology for integrating formal methods and software product line engineering to develop provably correct ACs for SPLs. To provide rigorous foundations for PLACIDUS, we define a variability-aware AC language and formalize its semantics using the proof assistant Lean. We provide tool support for PLACIDUS as part of an Eclipse-based model management framework. Finally, we demonstrate the feasibility of PLACIDUS by developing an AC for a product line of medical devices.

cs.SE↗

arXiv2024

A Software Engineering Perspective on Testing Large Language Models: Research, Practice, Tools and Benchmarks

Large Language Models (LLMs) are rapidly becoming ubiquitous both as stand-alone tools and as components of current and future software systems. To enable usage of LLMs in the high-stake or safety-critical systems of 2030, they need to undergo rigorous testing. Software Engineering (SE) research on testing Machine Learning (ML) components and ML-based systems has systematically explored many topics such as test input generation and robustness. We believe knowledge about tools, benchmarks, research and practitioner views related to LLM testing needs to be similarly organized. To this end, we present a taxonomy of LLM testing topics and conduct preliminary studies of state of the art and practice approaches to research, open-source tools and benchmarks for LLM testing, mapping results onto this taxonomy. Our goal is to identify gaps requiring more research and engineering effort and inspire a clearer communication between LLM practitioners and the SE research community.

cs.SE↗

arXiv2024

Normative Requirements Operationalization with Large Language Models

Normative non-functional requirements specify constraints that a system must observe in order to avoid violations of social, legal, ethical, empathetic, and cultural norms. As these requirements are typically defined by non-technical system stakeholders with different expertise and priorities (ethicists, lawyers, social scientists, etc.), ensuring their well-formedness and consistency is very challenging. Recent research has tackled this challenge using a domain-specific language to specify normative requirements as rules whose consistency can then be analysed with formal methods. In this paper, we propose a complementary approach that uses Large Language Models to extract semantic relationships between abstract representations of system capabilities. These relations, which are often assumed implicitly by non-technical stakeholders (e.g., based on common sense or domain knowledge), are then used to enrich the automated reasoning techniques for eliciting and analyzing the consistency of normative requirements. We show the effectiveness of our approach to normative requirements elicitation and operationalization through a range of real-world case studies.

cs.SE↗