Multimodal Plant Root Phenotyping with Integration of 3D Skeleton Extraction and Language Analysis

Abstract

Plant root phenotyping is fundamental to understanding below-ground structures, optimizing crop management, and improving agricultural sustainability. This paper presents a multimodal robotic AI framework that integrates 3D skeleton extraction with language-guided reasoning for interpretable and data-efficient root analysis. We develop an unsupervised skeleton extraction network based on Weighted Laplacian Contraction (W-LBC) to generate high-fidelity structural representations from dense point clouds captured by robotic 3D sensing platforms. Quantitative morphological descriptors, including root count, length, branching angle, and density, are computed from the reconstructed skeleton graph to capture geometric and topological characteristics. Building on these features, we introduce an Evidence-First language modeling framework that fine-tunes GPT as an interactive analytical chatbot using automatically generated instruction--response pairs. Each training sample provides measurable evidence before natural-language reasoning, enabling the model to ground interpretation in quantitative morphology. Through supervised fine-tuning, GPT associates numerical structure with semantic meaning, producing biologically consistent explanations of growth patterns and adaptive traits. Experiments show that the structure-guided framework achieves robust, interpretable reasoning across 12 plant species with diverse root architectures. By integrating unsupervised 3D geometric perception with large-scale language understanding, our approach bridges quantitative analysis and semantic interpretation, establishing a unified paradigm for explainable robotic plant root phenotyping.