Metrics for Benchmarking and Uncertainty Quantification: Quality, Applicability, and a Path to Best Practices for Machine Learning in Chemistry

Gaurav Vishwakarma; Aditya Sonpal; Johannes Hachmann

doi:10.1016/j.trechm.2020.12.004

Home/Archive/arXiv

arXiv · 2010.00110

Metrics for Benchmarking and Uncertainty Quantification: Quality, Applicability, and a Path to Best Practices for Machine Learning in Chemistry

Gaurav Vishwakarma·Aditya Sonpal·Johannes Hachmann

View original source ↗Read source document ↗Export metadata ↓

Abstract

This review aims to draw attention to two issues of concern when we set out to make machine learning work in the chemical and materials domain, i.e., statistical loss function metrics for the validation and benchmarking of data-derived models, and the uncertainty quantification of predictions made by them. They are often overlooked or underappreciated topics as chemists typically only have limited training in statistics. Aside from helping to assess the quality, reliability, and applicability of a given model, these metrics are also key to comparing the performance of different models and thus for developing guidelines and best practices for the successful application of machine learning in chemistry.

Explore related subjects

physics.chem-ph cs.LG

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

Gaurav Vishwakarma, Aditya Sonpal, Johannes Hachmann. 2021-01-22. Metrics for Benchmarking and Uncertainty Quantification: Quality, Applicability, and a Path to Best Practices for Machine Learning in Chemistry. https://doi.org/10.1016/j.trechm.2020.12.004

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Charge Transfer with a Spin. I: A Variational Constrained-CASSCF Framework for Investigating Charge Transfer in the Presence of Spin-Orbit Coupling

Charge transfer in open-shell molecular systems can involve a delicate interplay between charge localization, orbital relaxation, and spin-orbit coupling (SOC), particularly near ground-excited state crossings. Here, we introduce a very inexpensive variational framework for treating these effects in odd-electron systems by extending the electron/hole-transfer Dynamically-weighted State-Averaged Constrained CASSCF (eDSC/hDSC) method to include SOC. Our method incorporates the SOC Hamiltonian directly into the variational orbital optimization through complex-valued spinor orbitals, allowing orbital and spin degrees of freedom to relax self-consistently while preserving the time-reversal symmetry of doublet states. The method achieves smooth potential energy surfaces and rapid self-consistent field (SCF) convergence; moreover, from physically small to artificially large SOC strengths, the calculation can be converged to very tight thresholds across the whole potential energy surface. While the results presented here are for systems with two charge centers, as shown in the appendix, the theory is quite general and can be adapted to either a system of $N$ charge centers or a metal-molecule surface with a continuum of states, so that charge transport (and not just charge transfer) can also be studied. The approach therefore provides a route towards ab initio studies of spin-dependent charge transfer and charge transport in molecular systems with nontrivial spin degrees of freedom.

physics.chem-ph↗

arXiv2026

Charge Transfer with a Spin. II: A Framework for Diabatization which Localizes Charge and Spin

We investigate a diabatization procedure that localizes charges (in real space) and localizes spins (in spin space) for open-shell systems that exhibit charge transfer in the presence of spin-orbit coupling. The procedure is applied to a two-state crossing between pairs of Kramers-restricted doublet states (which can also be considered effectively a four-state crossing). To generate the relevant diabatic states, we employ a two-step optimization over complex-unitary rotations that sequentially maximizes dipole and spin moments through iterative Jacobi sweeps; the resulting update rules are effectively equivalent to those of approximate joint diagonalization (AJD) applied to charge and spin. The method converges rapidly and yields smooth diabatic potential energy surfaces that preserve dipole and spin properties (e.g., a smoothly varying spin quantization axis) along the reaction coordinate while maintaining time-reversal symmetry.

physics.chem-ph↗

arXiv2026

Fixed-Dimensional Latent Flow for Generating Variable-Size 3D Molecules

Molecular size is coupled to composition, structure, and function, yet most 3D molecular generators require a predefined atom count. We introduce Equivariant-Free Transformer-Autoencoded Latent Flow Matching, a two-stage framework that samples a fixed-dimensional latent vector using flow matching and uses an autoregressive Transformer to determine molecular size, atom types, coordinates, and chemical attributes. Canonical atom ordering and rigid-pose alignment enable Transformers without equivariant layers, while decoded attributes guide bond reconstruction. On PCQM4Mv2, unconditional generation yields 87.9\% unique, novel molecules passing sanitization and PoseBusters checks, exceeding baselines with lower end-to-end training and sampling time and higher end-to-end throughput. Across ten target HOMO-LUMO gaps, internal ranking retains 30\% of screened candidates and increases the density functional theory-verified hit rate within 0.1 eV from 25.0\% to 52.4\%, while largely preserving novelty and diversity. These results demonstrate fixed-dimensional latent generation with autoregressive decoding as a practical approach to molecular design without prespecifying size.

physics.chem-ph↗