Towards Sustainable Hydrogen Systems: Supply Chain Optimization with Model Predictive Control and Reinforcement Learning

Abstract

Hydrogen supply chains are expected to play a central role in future low-carbon energy systems by enabling renewable energy integration, long-duration storage, and decarbonization of industrial and transportation sectors. However, their operation is challenged by renewable generation variability, electricity price fluctuations, uncertain hydrogen demand, and engineering constraints associated with electrolyzers, energy storage, and grid interaction. As hydrogen infrastructure expands toward commercial deployment, operational strategies must balance economic performance, reliability, and sustainability under dynamic and uncertain conditions. This paper investigates and compares four control approaches for a renewable-powered hydrogen supply chain: a rule-based controller (RBC), model predictive control (MPC), reinforcement learning without forecasts (RL-NF), and reinforcement learning with forecast-augmented observations (RL-F). All methods are evaluated within a unified, physically realistic framework incorporating electrolyzer minimum-load and ramp-rate constraints, battery and hydrogen storage dynamics, grid import limits, and consistent economic assumptions, enabling a fair comparison under identical operating conditions. Simulation results show that MPC achieves the highest economic performance by exploiting short-term forecasts to coordinate storage, reduce grid dependence, and improve efficiency. RL-NF demonstrates robust and competitive performance without future information, highlighting the capability of learning-based methods to discover effective policies from experience. RL-F does not consistently outperform its no-forecast counterpart, suggesting that forecast uncertainty and increased state complexity can limit forecast-augmented learning. The results provide guidance for selecting operational control strategies in future hydrogen energy systems.