Dataversifying Natural Sciences: Pioneering a Data Lake Architecture for Curated Data-Centric Experiments in Life \& Earth Sciences

Genoveva Vargas-Solar; Jérôme Darmont; Alejandro Adorjan; Javier A. Espinosa-Oviedo; Carmem Hara; Sabine Loudcher; Regina Motz; Martin Musicante; José-Luis Zechinelli-Martini

arXiv · 2403.20063

Dataversifying Natural Sciences: Pioneering a Data Lake Architecture for Curated Data-Centric Experiments in Life \& Earth Sciences

Genoveva Vargas-Solar·Jérôme Darmont·Alejandro Adorjan·Javier A. Espinosa-Oviedo·Carmem Hara·Sabine Loudcher·Regina Motz·Martin Musicante·José-Luis Zechinelli-Martini

View original source ↗Read source document ↗Export metadata ↓

Abstract

This vision paper introduces a pioneering data lake architecture designed to meet Life \& Earth sciences' burgeoning data management needs. As the data landscape evolves, the imperative to navigate and maximize scientific opportunities has never been greater. Our vision paper outlines a strategic approach to unify and integrate diverse datasets, aiming to cultivate a collaborative space conducive to scientific discovery.The core of the design and construction of a data lake is the development of formal and semi-automatic tools, enabling the meticulous curation of quantitative and qualitative data from experiments. Our unique ''research-in-the-loop'' methodology ensures that scientists across various disciplines are integrally involved in the curation process, combining automated, mathematical, and manual tasks to address complex problems, from seismic detection to biodiversity studies. By fostering reproducibility and applicability of research, our approach enhances the integrity and impact of scientific experiments. This initiative is set to improve data management practices, strengthening the capacity of Life \& Earth sciences to solve some of our time's most critical environmental and biological challenges.

Explore related subjects

cs.DB

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

Genoveva Vargas-Solar, Jérôme Darmont, Alejandro Adorjan, Javier A. Espinosa-Oviedo, Carmem Hara, Sabine Loudcher, Regina Motz, Martin Musicante, José-Luis Zechinelli-Martini. 2024-03-29. Dataversifying Natural Sciences: Pioneering a Data Lake Architecture for Curated Data-Centric Experiments in Life \& Earth Sciences. https://arxiv.org/abs/2403.20063

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Towards Effective Orchestration of AI x DB Workloads

AI-driven analytics are increasingly crucial to data-centric decision-making. Executing relational and AI operators in separate runtimes prevents the database optimizer and runtime from coordinating operator ordering, model placement, batching, and state reuse. Integrating AI operators into database engines enables such coordination but raises challenges in jointly optimizing query processing and model execution, scheduling under resource contention, and reusing relational intermediates and AI artifacts. This paper formalizes AIxDB workloads as iterative, concurrent, and shareable executions that interleave relational and AI operators. We then advocate database-native orchestration as a paradigm for redesigning database engines for these workloads and distill two design principles: holistic AIxDB co-optimization and unified AIxDB cache management. We present NeurEngine as a proof-of-concept prototype and report preliminary results illustrating the performance benefits of database-native orchestration

cs.DB↗

arXiv2026

A Functional Pilot for Certified Freshness-Aware Semantic--Spatial Range Retrieval

Geographic applications need every object inside a radius that satisfies a semantic threshold, yet embedding indexes return approximate top-ranked lists and may omit qualifying records silently. We present FRESH-GEORANGE, a semantic- spatial range design that separates source-watermark freshness from optional record age. Geographic cells and semantic mi- croblocks provide admissible pruning bounds; a graph proposes verification order but supplies no correctness evidence. Exact mode scans every nonprunable block and the delta overlay. Certified mode may stop early and reports a deterministic query- specific recall lower bound from verified answers and unresolved records. A reproducible CPU pilot uses 2,500 real OpenFlights airport records, a 2,000-record base, and 740 simulated insert, delete, and text-revision events; it evaluates 180 unique queries over five seeds. Exact mode achieved 100.00% set recall on every query. The 95-percent mode achieved 99.91% empirical mean recall with a 99.41% reported mean certificate and no observed bound violation. However, its 7.24 ms median latency was 5.85 times the 1.24 ms spatial-first exact baseline, and full-history delta replay became slower than rebuilding at larger batches. The prototype therefore validates the completeness mechanism, not performance superiority or production freshness. Submission- scale evaluation requires real map diffs, official recent baselines, and truly incremental versioned maintenance.

cs.DB↗

arXiv2026

AkasicMEM: Governed Enterprise Memory for Agents

Agent memory enables enterprise agents to retain knowledge acquired during work and reuse it across tasks and agents, turning execution experience into persistent organizational knowledge. Realizing this potential requires both source--memory integration, through which enterprise sources and accumulated memory can be utilized together, and memory governance, through which shared memory remains subject to organizational policies throughout its lifecycle. These requirements interact when information from enterprise sources persists in memory. As this information is repeatedly derived and reused under changing principals and policies, source restrictions may be bypassed, resulting in information leakage. Preventing such leakage requires authorization continuity, under which source restrictions remain effective throughout source-to-memory and memory-to-memory derivation and reuse. Existing approaches address these concerns individually, but do not treat source--memory integration, memory governance, and authorization continuity as combined core design targets across the memory lifecycle. We define Governed Enterprise Memory as agent memory designed around this combined scope and present AkasicMEM as its realization. AkasicMEM realizes authorization continuity through transitive lineage, policy composition during memory formation, and policy re-evaluation during retrieval. It is built on GraphAI's AkasicDB, a unified vector--graph--relational database whose storage and execution substrate enables the underlying operations of these mechanisms to be jointly optimized and executed.

cs.DB↗