NovaFilter - A Rule-Based ETL Staging Framework for Fault-Tolerant Relational Databases

Meike Klettke

Home/Archive/arXiv

arXiv · 2611.00959

NovaFilter - A Rule-Based ETL Staging Framework for Fault-Tolerant Relational Databases

Meike Klettke

View original source ↗Read source document ↗Export metadata ↓

Abstract

NovaFilter is a completely new approach to deriving structured relational data from heterogeneous input data in different data formats. It combines consistency checks, rule-based data transformation, and a human-in-the-loop example discussion. Using these three components, NovaFilter stores all transformation knowledge in an explicit rule base, which is continuously expanded through the example discussion with the user. The resulting data transformation process is deterministic, transparent, and reusable. As a result, the method can replace conventional ETL scripts and offers a completely new strategy for transforming data between different data models and schemas. Its main advantage is that users without programming skills (e.g. domain experts) can answer the example discussion and thus it is a building block for democratization of ETL script developement.

Explore related subjects

cs.DB

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

Meike Klettke. 2026-10-31. NovaFilter - A Rule-Based ETL Staging Framework for Fault-Tolerant Relational Databases. https://arxiv.org/abs/2611.00959

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

How to Define the Quality of Data? A Feature-Based Literature Survey

Data is generated in almost every digital interaction, and using it effectively requires high quality. But what exactly is data quality? The literature offers many, widely diverging answers. This article presents a systematic literature review that screened more than 8,700 records on data quality and identified 35 providing original, dimension-based definitions. Using feature-oriented domain analysis, a feature model of these heterogeneous definitions is derived and used to classify all identified publications. The classification reveals a heterogeneous landscape: despite an ISO standard and a few highly cited quasi-standards, no consensus exists on which quality dimensions to consider or how to define them. As definitions increasingly emerge for specific domains, contexts of use, and new types of data, the respective communities need to deliberately converge on how data quality is defined in their field.

cs.DB↗

arXiv2026

Navigating Runtime Dynamism in Semantic Query Serving

LLM-powered semantic queries increasingly combine relational processing with multiple LLM-based operators over unstructured data. Efficiently serving such queries is challenging due to runtime dynamism at two coupled levels. Across queries, dynamic arrivals and heterogeneous resource demands continuously change scheduling priorities. Within a query, heterogeneous characteristics across fine-grained input batches cause operator costs and selectivities to vary throughout execution, making a statically optimized plan unreliable. These two forms of dynamism create a circular dependency: inter-query scheduling requires estimates of remaining LLM work, while the remaining work depends on runtime plan choices whose costs become observable only during execution. We present SeqoFlow, a runtime framework that jointly adapts query plans and schedules concurrent semantic queries. SeqoFlow enumerates equivalence-preserving plan alternatives under operator dependencies and decomposes queries into fine-grained units, enabling safe plan adaptation and query preemption at batch boundaries. To support both decisions without exhaustively profiling candidate plans on the target LLM, SeqoFlow uses lightweight speculative estimation to capture runtime costs. These estimates jointly guide plan adaptation and inter-query scheduling, closing the loop between query optimization and LLM inference. Experiments across diverse query structures, data distributions, and arrival rates show that SeqoFlow substantially reduces end-to-end latency compared to existing semantic query systems, adaptive query processing techniques, and conventional scheduling policies.

cs.DB↗

arXiv2026

Text-to-SQL Models Make Choices Without Saying So

A text-to-SQL answer can run, return a plausible number and still compute something other than what the analyst meant. This can happen when the request leaves a calculation choice open, such as whether to count rows or distinct entities, and the model settles it without saying so. Standard benchmarks cannot detect this, because they score each question against a single reference interpretation. We introduce SpecBridge, which tests such a choice by varying it alone: it poses three otherwise identical requests, one leaving the choice open and one per interpretation, and scores each answer by running it. The databases are real, and the choices are ones that made models fail on existing benchmarks. On 204 hand-validated tests, four language models usually compute each interpretation when asked for it, yet for the open request return both in only 15-58% of the tests where they can, often without mentioning the choice. The gap lies in reporting, not in computing. Benchmarks even penalize the alternative: on BIRD questions where such a choice changes the answer, over two in five of the answers marked wrong are right under the other interpretation. Because the gap lies in reporting, a system can narrow it without a stronger model: it first decides whether a request leaves a choice open, then answers each interpretation in its own call. On our tests and on AMBROSIA, an existing benchmark of ambiguous questions, this approach raises the share of open requests that get every interpretation by 29-63 points over the single answer that production assistants give, without lowering precision on clear requests. It also needs fewer user turns than asking a clarifying question.

cs.DB↗