arXiv subjects

Jean-Rémy Falleri

Publications and source records attributed to Jean-Rémy Falleri.

At least 19 recordsLinked to original sources

arXiv2026

DRAGON: Robust Classification for Very Large Collections of Software Repositories

The ability to automatically classify source code repositories with ''topics'' that reflect their content and purpose is very useful, especially when navigating or searching through large software collections. However, existing approaches often rely heavily on README files and other metadata, which are frequently missing, limiting their applicability in real-world large-scale settings. We present DRAGON, a repository classifier designed for very large and diverse software collections. It operates entirely on lightweight signals commonly stored in version control systems: file and directory names, and optionally the README when available. In repository classification at scale, DRAGON improves F1@5 from 54.8% to 60.8%, surpassing the state of the art. DRAGON remains effective even when README files are absent, with performance degrading by only 6% w.r.t. when they are present. This robustness makes it practical for real-world settings where documentation is sparse or inconsistent. Furthermore, many of the remaining classification errors are near misses, where predicted labels are semantically close to the correct topics. This property increases the practical value of the predictions in real-world software collections, where suggesting a few related topics can still guide search and discovery. As a byproduct of developing DRAGON, we also release the largest open dataset to date for repository classification, consisting of 825 thousand repositories with associated ground-truth topics, sourced from the Software Heritage archive, providing a foundation for future large-scale and language-agnostic research on software repository understanding.

cs.SE↗

arXiv2025

Client--Library Compatibility Testing with API Interaction Snapshots

Modern software development heavily relies on third-party libraries to speed up development and enhance quality. As libraries evolve, they may break the tacit contract established with their clients by introducing behavioral breaking changes (BBCs) that alter run-time behavior and silently break client applications without being detected at compile time. Traditional regression tests on the client side often fail to detect such BBCs, either due to limited library coverage or weak assertions that do not sufficiently exercise the library's expected behavior. To address this issue, we propose a novel approach to client--library compatibility testing that leverages existing client tests in a novel way. Instead of relying on developer-written assertions, we propose recording the actual interactions at the API boundary during the execution of client tests (protocol, input and output values, exceptions, etc.). These sequences of API interactions are stored as snapshots which capture the exact contract expected by a client at a specific point in time. As the library evolves, we compare the original and new snapshots to identify perturbations in the contract, flag potential BBCs, and notify clients. We implement this technique in our prototype tool Gilesi, a Java framework that automatically instruments library APIs, records snapshots, and compares them. Through a preliminary case study on several client--library pairs with artificially seeded BBCs, we show that Gilesi reliably detects BBCs missed by client test suites.

cs.SE↗

arXiv2025

Roseau: Fast, Accurate, Source-based API Breaking Change Analysis in Java

Understanding API evolution and the introduction of breaking changes (BCs) in software libraries is essential for library maintainers to manage backward compatibility and for researchers to conduct empirical studies on software library evolution. In Java, tools such as JApiCmp and Revapi are commonly used to detect BCs between library releases, but their reliance on binary JARs limits their applicability. This restriction hinders large-scale longitudinal studies of API evolution and fine-grained analyses such as commit-level BC detection. In this paper, we introduce Roseau, a novel static analysis tool that constructs technology-agnostic API models from library code equipped with rich semantic analyses. API models can be analyzed to study API evolution and compared to identify BCs between any two versions of a library (releases, commits, branches, etc.). Unlike traditional approaches, Roseau can build API models from source code or bytecode, and is optimized for large-scale longitudinal analyses of library histories. We assess the accuracy, performance, and suitability of Roseau for longitudinal studies of API evolution, using JApiCmp and Revapi as baselines. We extend and refine an established benchmark of BCs and show that Roseau achieves higher accuracy (F1 = 0.99) than JApiCmp (F1 = 0.86) and Revapi (F1 = 0.91). We analyze 60 popular libraries from Maven Central and find that Roseau delivers excellent performance, detecting BCs between versions in under two seconds, including in libraries with hundreds of thousands of lines of code. We further illustrate the limitations of JApiCmp and Revapi for longitudinal studies and the novel analysis capabilities offered by Roseau by tracking the evolution of Google's Guava API and the introduction of BCs over 14 years and 6,839 commits, reducing analysis times from a few days to a few minutes.

cs.SE↗

arXiv2025

What Happened in This Pipeline? Diffing Build Logs with CiDiff

Continuous integration (CI) is widely used by developers to ensure the quality and reliability of their software projects. However, diagnosing a CI regression is a tedious process that involves the manual analysis of lengthy build logs. In this paper, we explore how textual differencing can support the debugging of CI regressions. As off-the-shelf diff algorithms produce suboptimal results, in this work we introduce a new diff algorithm specifically tailored to build logs called CiDiff. We evaluate CiDiff against several baselines on a novel dataset of 17 906 CI regressions, performing an accuracy study, a quantitative study and a user-study. Notably, our algorithm reduces the number of lines to inspect by about 60 % in the median case, with reasonable overhead compared to the state-of-practice LCS-diff. Finally, our algorithm is preferred by the majority of participants in 70 % of the regression cases, whereas LCS-diff is preferred in only 5 % of the cases.

cs.SE↗

arXiv2025

Are Autonomous Web Agents Good Testers?

Despite advances in automated testing, manual testing remains prevalent due to the high maintenance demands associated with test script fragility-scripts often break with minor changes in application structure. Recent developments in Large Language Models (LLMs) offer a potential alternative by powering Autonomous Web Agents (AWAs) that can autonomously interact with applications. These agents may serve as Autonomous Test Agents (ATAs), potentially reducing the need for maintenance-heavy automated scripts by utilising natural language instructions similar to those used by human testers. This paper investigates the feasibility of adapting AWAs for natural language test case execution and how to evaluate them. We contribute with (1) a benchmark of three offline web applications, and a suite of 113 manual test cases, split between passing and failing cases, to evaluate and compare ATAs performance, (2) SeeAct-ATA and pinATA, two open-source ATA implementations capable of executing test steps, verifying assertions and giving verdicts, and (3) comparative experiments using our benchmark that quantifies our ATAs effectiveness. Finally we also proceed to a qualitative evaluation to identify the limitations of PinATA, our best performing implementation. Our findings reveal that our simple implementation, SeeAct-ATA, does not perform well compared to our more advanced PinATA implementation when executing test cases (50% performance improvement). However, while PinATA obtains around 60% of correct verdict and up to a promising 94% specificity, we identify several limitations that need to be addressed to develop more resilient and reliable ATAs, paving the way for robust, low maintenance test automation. CCS Concepts: $\bullet$ Software and its engineering $\rightarrow$ Software testing and debugging.

cs.SE↗

arXiv2024

Lightweight Syntactic API Usage Analysis with UCov

Designing an effective API is essential for library developers as it is the lens through which clients will judge its usability and benefits, as well as the main friction point when the library evolves. Despite its importance, defining the boundaries of an API is a challenging task, mainly due to the diverse mechanisms provided by programming languages that have non-trivial interplays. In this paper, we present a novel conceptual framework designed to assist library maintainers in understanding the interactions allowed by their APIs via the use of syntactic usage models. These customizable models enable library maintainers to improve their design ahead of release, reducing friction during evolution. The complementary syntactic usage footprints and coverage scores, inferred from client code using the API (e.g., documentation samples, tests, third-party clients), enable developers to understand in-the-wild uses of their APIs and to reflect on the adequacy of their tests and documentation. We implement these models for Java libraries in a new tool UCov and demonstrate its capabilities on three libraries exhibiting diverse styles of interaction: jsoup, commons-cli, and spark. Our exploratory case study shows that UCov provides valuable information regarding API design and fine-grained analysis of client code to identify under-tested and under-documented library code.

cs.SE↗

arXiv2024

What the Fix? A Study of ASATs Rule Documentation

Automatic Static Analysis Tools (ASATs) are widely used by software developers to diffuse and enforce coding practices. Yet, we know little about the documentation of ASATs, despite it being critical to learn about the coding practices in the first place. We shed light on this through several contributions. First, we analyze the documentation of more than 100 rules of 16 ASATs for multiple programming languages, and distill a taxonomy of the purposes of the documentation-What triggers a rule; Why it is important; and how to Fix an issue-and its types of contents. Then, we conduct a survey to assess the effectiveness of the documentation in terms of its goals and types of content. We highlight opportunities for improvement in ASAT documentation. In particular, we find that the Why purpose is missing in half of the rules we survey; moreover, when the Why is present, it is more likely to have quality issues than the What and the Fix.

cs.SE↗

arXiv2023

MOON: Assisting Students in Completing Educational Notebook Scenarios

Jupyter notebooks are increasingly being adopted by teachers to deliver interactive practical sessions to their students. Notebooks come with many attractive features, such as the ability to combine textual explanations, multimedia content, and executable code alongside a flexible execution model which encourages experimentation and exploration. However, this execution model can quickly become an issue when students do not follow the intended execution order of the teacher, leading to errors or misleading results that hinder their learning. To counter this adverse effect, teachers usually write detailed instructions about how students are expected to use the notebooks. Yet, the use of digital media is known to decrease reading efficiency and compliance with written instructions, resulting in frequent notebook misuse and students getting lost during practical sessions. In this article, we present a novel approach, MOON, designed to remedy this problem. The central idea is to provide teachers with a language that enables them to formalize the expected usage of their notebooks in the form of a script and to interpret this script to guide students with visual indications in real time while they interact with the notebooks. We evaluate our approach using a randomized controlled experiment involving 21 students, which shows that MOON helps students comply better with the intended scenario without hindering their ability to progress. Our follow-up user study shows that about 75% of the surveyed students perceived MOON as rather useful or very useful.

cs.CY↗

arXiv2023

MLinter: Learning Coding Practices from Examples-Dream or Reality?

Coding practices are increasingly used by software companies. Their use promotes consistency, readability, and maintainability, which contribute to software quality. Coding practices were initially enforced by general-purpose linters, but companies now tend to design and adopt their own company-specific practices. However, these company-specific practices are often not automated, making it challenging to ensure they are shared and used by developers. Converting these practices into linter rules is a complex task that requires extensive static analysis and language engineering expertise. In this paper, we seek to answer the following question: can coding practices be learned automatically from examples manually tagged by developers? We conduct a feasibility study using CodeBERT, a state-of-the-art machine learning approach, to learn linter rules. Our results show that, although the resulting classifiers reach high precision and recall scores when evaluated on balanced synthetic datasets, their application on real-world, unbalanced codebases, while maintaining excellent recall, suffers from a severe drop in precision that hinders their usability.

cs.SE↗

arXiv2022

Spork: Structured Merge for Java with Formatting Preservation

The highly parallel workflows of modern software development have made merging of source code a common activity for developers. The state of the practice is based on line-based merge, which is ubiquitously used with "git merge". Line-based merge is however a generalized technique for any text that cannot leverage the structured nature of source code, making merge conflicts a common occurrence. As a remedy, research has proposed structured merge tools, which typically operate on abstract syntax trees instead of raw text. Structured merging greatly reduces the prevalence of merge conflicts but suffers from important limitations, the main ones being a tendency to alter the formatting of the merged code and being prone to excessive running times. In this paper, we present SPORK, a novel structured merge tool for JAVA. SPORK is unique as it preserves formatting to a significantly greater degree than comparable state-of-the-art tools. SPORK is also overall faster than the state of the art, in particular significantly reducing worst-case running times in practice. We demonstrate these properties by replaying 1740 real-world file merges collected from 119 open-source projects, and further demonstrate several key differences between SPORK and the state of the art with in-depth case studies.

cs.SE↗

arXiv2021

BreakBot: Analyzing the Impact of Breaking Changes to Assist Library Evolution

"If we make this change to our code, how will it impact our clients?" It is difficult for library maintainers to answer this simple-yet essential!-question when evolving their libraries. Library maintainers are constantly balancing between two opposing positions: make changes at the risk of breaking some of their clients, or avoid changes and maintain compatibility at the cost of immobility and growing technical debt. We argue that the lack of objective usage data and tool support leaves maintainers with their own subjective perception of their community to make these decisions. We introduce BreakBot, a bot that analyses the pull requests of Java libraries on GitHub to identify the breaking changes they introduce and their impact on client projects. Through static analysis of libraries and clients, it extracts and summarizes objective data that enrich the code review process by providing maintainers with the appropriate information to decide whether-and how-changes should be accepted, directly in the pull requests.

cs.SE↗

arXiv2021

Breaking Bad? Semantic Versioning and Impact of Breaking Changes in Maven Central

Just like any software, libraries evolve to incorporate new features, bug fixes, security patches, and refactorings. However, when a library evolves, it may break the contract previously established with its clients by introducing Breaking Changes (BCs) in its API. These changes might trigger compile-time, link-time, or run-time errors in client code. As a result, clients may hesitate to upgrade their dependencies, raising security concerns and making future upgrades even more difficult.Understanding how libraries evolve helps client developers to know which changes to expect and where to expect them, and library developers to understand how they might impact their clients. In the most extensive study to date, Raemaekers et al. investigate to what extent developers of Java libraries hosted on the Maven Central Repository (MCR) follow semantic versioning conventions to signal the introduction of BCs and how these changes impact client projects. Their results suggest that BCs are widespread without regard for semantic versioning, with a significant impact on clients.In this paper, we conduct an external and differentiated replication study of their work. We identify and address some limitations of the original protocol and expand the analysis to a new corpus spanning seven more years of the MCR. We also present a novel static analysis tool for Java bytecode, Maracas, which provides us with: (i) the set of all BCs between two versions of a library; and (ii) the set of locations in client code impacted by individual BCs. Our key findings, derived from the analysis of 119, 879 library upgrades and 293, 817 clients, contrast with the original study and show that 83.4% of these upgrades do comply with semantic versioning. Furthermore, we observe that the tendency to comply with semantic versioning has significantly increased over time. Finally, we find that most BCs affect code that is not used by any client, and that only 7.9% of all clients are affected by BCs. These findings should help (i) library developers to understand and anticipate the impact of their changes; (ii) library users to estimate library upgrading effort and to pick libraries that are less likely to break; and (iii) researchers to better understand the dynamics of library-client co-evolution in Java.

cs.SE↗

arXiv2021

A grounded theory of Community Package Maintenance Organizations-Registered Report

a) Context: In many programming language ecosystems, developers rely more and more on external open source dependencies, made available through package managers. Key ecosystem packages that go unmaintained create a health risk for the projects that depend on them and for the ecosystem as a whole. Therefore, community initiatives can emerge to alleviate the problem by adopting packages in need of maintenance. b) Objective: The goal of our study is to explore such community initiatives, that we will designate from now on as Community Package Maintenance Organizations (CPMOs) and to build a theory of how and why they emerge, how they function and their impact on the surrounding ecosystems. c) Method: To achieve this, we plan on using a qualitative methodology called Grounded Theory. We have begun applying this methodology, by relying on "extant" documents originating from several CPMOs. We present our preliminary results and the research questions that have emerged. We plan to answer these questions by collecting appropriate data (theoretical sampling), in particular by contacting CPMO participants and questioning them by e-mails, questionnaires or semi-structured interviews. d) Impact: Our theory should inform developers willing to launch a CPMO in their own ecosystem and help current CPMO participants to better understand the state of the practice and what they could do better.

cs.SE↗

arXiv2021

Can We Spot Energy Regressions using Developers Tests?

Software Energy Consumption(SEC) is gaining more and more attention. In this paper, we tackle the problem of hinting developers about the SEC of their programs in the context of software developments based on Continuous Integration(CI). In this study, we investigate if the CI can leverage developers' tests to perform a new class of tests: the energy regression testing. Energy regression is similar to performance regression but focused on the energy consumption of the program instead of standard performance indicators, like execution time or memory consumption. We propose to perform an exploratory study of the usage of developers' tests for energy regression testing. We propose to first investigate if developers' tests can be used to obtain stable SEC indicators. Then, to consider if comparing the SEC of developers' tests between two versions can accurately spot energy regressions introduced by automated program mutations. Finally, to assess if it can successfully pinpoint the source code lines guilty of energy regressions. Our study will pave the way for automated SEC regression tools that can be readily deployed inside an existing CI infrastructure to raise awareness of SEC issues among practitioners.

cs.SE↗

arXiv2021

Assessment of a hybrid software development process for student projects: a controlled experiment

In recent years, a vivid interest in hybrid development methods has been observed as practitioners combine various approaches to software creation to improve productivity, product quality, and adaptability of the process to react to change. Scientific papers on the subject proliferate, however evaluation of the effectiveness of hybrid methods in academic contexts has yet to follow. The work presented investigates if introducing a hybrid approach for student projects brings added value as compared to iterative and sequential development. A controlled experiment was carried out among Bachelor students of a French engineering school to assess the impacts of a given development method on the success of student computing undertakings. Its three dimensions were examined via a set of metrics: product quality, team productivity as well as human factors (teamwork quality & learning outcomes). Several patterns were observed, which can provide a starting point for educators and researchers wishing to tailor or design a software development process for academic needs.

cs.SE↗

arXiv2020

Hyperparameter Optimization for AST Differencing

Computing the differences between two versions of the same program is an essential task for software development and software evolution research. AST differencing is the most advanced way of doing so, and an active research area. Yet, AST differencing algorithms rely on configuration parameters that may have a strong impact on their effectiveness. In this paper, we present a novel approach named DAT (Diff Auto Tuning) for hyperparameter optimization of AST differencing. We thoroughly state the problem of hyper-configuration for AST differencing. We evaluate our data-driven approach DAT to optimize the edit-scripts generated by the state-of-the-art AST differencing algorithm named GumTree in different scenarios. DAT is able to find a new configuration for GumTree that improves the edit-scripts in 21.8% of the evaluated cases.

cs.SE↗

arXiv2018

Gamification: a Game Changer for Managing Technical Debt? A Design Study

Context: Technical debt management is challenging for software engineers due to poor tool support and a lack of knowledge on how to prioritize technical debt repayment and prevention activities. Furthermore, when there is a large backlog of debt, developers often lack the motivation to address it. Objective: In this paper, we describe a design study to investigate how gamification can support Technical Debt Management in a large legacy software system of an industrial company. Our study leads to a novel tool (named Themis) that combines technical debt support, version control, and gamification features. In addition to gamification features, Themis provides suggestions for developers on where to focus their effort, and visualizations for managers to track technical debt activities. Method: We describe how Themis was refined and validated in an iterative deployment with the company, finally conducting a qualitative study to investigate how the features of Themis affect technical debt management behavior. We consider the impact on both developers and managers. Results: Our results show that it achieves increased developer motivation, and supports managers in monitoring and influencing developer behaviors. We show how our findings may be transferable to other contexts by proposing guidelines on how to apply gamification. Conclusions: With this case, gamification appears as a promising solution to help technical debt management, although it needs to be carefully designed and implemented to avoid its possible negative effects.

cs.SE↗

arXiv2013

The Harmony Platform

According to Wikipedia, The Mining Software Repositories (MSR) field analyzes the rich data available in software repositories, such as version control repositories, mailing list archives, bug tracking systems, issue tracking systems, etc. to uncover interesting and actionable information about software systems, projects and software engineering. The MSR field has received a great deal of attention and has now its own research conference : http://www.msrconf.org/. However performing MSR studies is still a technical challenge. Indeed, data sources (such as version control system or bug tracking systems) are highly heterogeneous. Moreover performing a study on a lot of data sources is very expensive in terms of execution time. Surprisingly, there are not so many tools able to help researchers in their MSR quests. This is why we created the Harmony platform, as a mean to assist researchers in performing MSR studies.

cs.SE↗