Flow with FlorDB: Incremental Context Maintenance for the Machine Learning Lifecycle

Rolando Garcia; Pragya Kallanagoudar; Chithra Anand; Sarah E. Chasins; Joseph M. Hellerstein; Erin Michelle Turner Kerrison; Aditya G. Parameswaran

arXiv · 2408.02498

Flow with FlorDB: Incremental Context Maintenance for the Machine Learning Lifecycle

Rolando Garcia·Pragya Kallanagoudar·Chithra Anand·Sarah E. Chasins·Joseph M. Hellerstein·Erin Michelle Turner Kerrison·Aditya G. Parameswaran

View original source ↗Read source document ↗Export metadata ↓

Abstract

In this paper we present techniques to incrementally harvest and query arbitrary metadata from machine learning pipelines, without disrupting agile practices. We center our approach on the developer-favored technique for generating metadata -- log statements -- leveraging the fact that logging creates context. We show how hindsight logging allows such statements to be added and executed post-hoc, without requiring developer foresight. Relational views of incomplete metadata can be queried to dynamically materialize new metadata in bulk and on demand across multiple versions of workflows. This is done in a "metadata later" style, off the critical path of agile development. We realize these ideas in a system called FlorDB and demonstrate how the data context framework covers a range of both ad-hoc metadata as well as special cases treated today by bespoke feature stores and model repositories. Through a usage scenario -- including both ML and human feedback -- we illustrate how the component techniques come together to resolve classic software engineering trade-offs between agility and discipline.

Explore related subjects

cs.DB cs.SE

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

Rolando Garcia, Pragya Kallanagoudar, Chithra Anand, Sarah E. Chasins, Joseph M. Hellerstein, Erin Michelle Turner Kerrison, Aditya G. Parameswaran. 2024-11-15. Flow with FlorDB: Incremental Context Maintenance for the Machine Learning Lifecycle. https://arxiv.org/abs/2408.02498

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Towards Effective Orchestration of AI x DB Workloads

AI-driven analytics are increasingly crucial to data-centric decision-making. Executing relational and AI operators in separate runtimes prevents the database optimizer and runtime from coordinating operator ordering, model placement, batching, and state reuse. Integrating AI operators into database engines enables such coordination but raises challenges in jointly optimizing query processing and model execution, scheduling under resource contention, and reusing relational intermediates and AI artifacts. This paper formalizes AIxDB workloads as iterative, concurrent, and shareable executions that interleave relational and AI operators. We then advocate database-native orchestration as a paradigm for redesigning database engines for these workloads and distill two design principles: holistic AIxDB co-optimization and unified AIxDB cache management. We present NeurEngine as a proof-of-concept prototype and report preliminary results illustrating the performance benefits of database-native orchestration

cs.DB↗

arXiv2026

Benchmarking Text-to-SQL under Role-Based Access Control

Given a database S and a natural language question Q, text-to-SQL systems aim to generate an SQL query that correctly answers Q when executed against S. Currently, popular text-to-SQL benchmarks mostly assume unrestricted access to S; in practice, however, user access is often restricted, e.g., through role-based access control (RBAC) policies. This leads to a potential disconnect between benchmarking results and real-world performance: an LLM with high benchmark scores might perform poorly in an access-controlled environment, by frequently violating RBAC, or rejecting a query q that could be answered with only permitted data in S. Motivated by this, we present a comprehensive text-to-SQL benchmarking framework with realistic RBAC constraints, which features an LLM-assisted workflow that augments existing text-to-SQL benchmarks with plausible user roles and access policies. To do so, we formulate the problem of role synthesis as a structured reasoning process over the database schema, in which the LLM first infers the application context from the schema, and then derives role responsibilities and access scopes consistent with this context. This process is audited by human-in-the-loop quality control, in which domain experts perform metric-guided screening on the generated roles. Besides the augmented dataset, the proposed framework also contains evaluation metrics that identify RBAC-specific failure modes, and disentangle SQL utility from access-control compliance. We apply the proposed framework to several widely-used benchmarks, and conduct a systematic empirical study of state-of-the-art text-to-SQL systems. The results show that many solutions (especially open-weight LLMs) with high benchmarking scores under an unrestricted setting suffer sharp performance degradation once access constraints are in place, due to frequent RBAC violations.

cs.DB↗

arXiv2026

A Functional Pilot for Certified Freshness-Aware Semantic--Spatial Range Retrieval

Geographic applications need every object inside a radius that satisfies a semantic threshold, yet embedding indexes return approximate top-ranked lists and may omit qualifying records silently. We present FRESH-GEORANGE, a semantic- spatial range design that separates source-watermark freshness from optional record age. Geographic cells and semantic mi- croblocks provide admissible pruning bounds; a graph proposes verification order but supplies no correctness evidence. Exact mode scans every nonprunable block and the delta overlay. Certified mode may stop early and reports a deterministic query- specific recall lower bound from verified answers and unresolved records. A reproducible CPU pilot uses 2,500 real OpenFlights airport records, a 2,000-record base, and 740 simulated insert, delete, and text-revision events; it evaluates 180 unique queries over five seeds. Exact mode achieved 100.00% set recall on every query. The 95-percent mode achieved 99.91% empirical mean recall with a 99.41% reported mean certificate and no observed bound violation. However, its 7.24 ms median latency was 5.85 times the 1.24 ms spatial-first exact baseline, and full-history delta replay became slower than rebuilding at larger batches. The prototype therefore validates the completeness mechanism, not performance superiority or production freshness. Submission- scale evaluation requires real map diffs, official recent baselines, and truly incremental versioned maintenance.

cs.DB↗