Computational Discovery of Energy-Efficient Heat Treatment for Microstructure Design using Deep Reinforcement Learning

Jaber R. Mianroodi; Nima H. Siboni; Dierk Raabe

Home/Archive/arXiv

arXiv · 2209.11259

Computational Discovery of Energy-Efficient Heat Treatment for Microstructure Design using Deep Reinforcement Learning

Jaber R. Mianroodi·Nima H. Siboni·Dierk Raabe

View original source ↗Read source document ↗Export metadata ↓

Abstract

Deep Reinforcement Learning (DRL) is employed to develop autonomously optimized and custom-designed heat-treatment processes that are both, microstructure-sensitive and energy efficient. Different from conventional supervised machine learning, DRL does not rely on static neural network training from data alone, but a learning agent autonomously develops optimal solutions, based on reward and penalty elements, with reduced or no supervision. In our approach, a temperature-dependent Allen-Cahn model for phase transformation is used as the environment for the DRL agent, serving as the model world in which it gains experience and takes autonomous decisions. The agent of the DRL algorithm is controlling the temperature of the system, as a model furnace for heat-treatment of alloys. Microstructure goals are defined for the agent based on the desired microstructure of the phases. After training, the agent can generate temperature-time profiles for a variety of initial microstructure states to reach the final desired microstructure state. The agent's performance and the physical meaning of the heat-treatment profiles generated are investigated in detail. In particular, the agent is capable of controlling the temperature to reach the desired microstructure starting from a variety of initial conditions. This capability of the agent in handling a variety of conditions paves the way for using such an approach also for recycling-oriented heat treatment process design where the initial composition can vary from batch to batch, due to impurity intrusion, and also for the design of energy-efficient heat treatments. For testing this hypothesis, an agent without penalty on the total consumed energy is compared with one that considers energy costs. The energy cost penalty is imposed as an additional criterion on the agent for finding the optimal temperature-time profile.

Explore related subjects

cond-mat.mtrl-sci cs.LG

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

Jaber R. Mianroodi, Nima H. Siboni, Dierk Raabe. 2022-09-22. Computational Discovery of Energy-Efficient Heat Treatment for Microstructure Design using Deep Reinforcement Learning. https://arxiv.org/abs/2209.11259

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Incommensurate structural and magnetic modulations in potassium-rich cryptomelane, K$_x$Mn$_8$O$_{16}$ ($x\approx1.45$)

Cryptomelane is a hollandite-like material consisting of K$^+$ cations in an $α$-MnO$_2$ tunnel-like crystallographic motif. Cryptomelane with stoichiometry K$_x$Mn$_8$O$_{16}$ ($x\approx1.45$) has been synthesized and its magnetic properties investigated using variable-temperature magnetic susceptibility, heat capacity, and neutron powder diffraction. Three distinct transitions at $T_1=184$\,K, $T_2=54.5$\,K, and $T_3=24$\,K are observed. At $T_1$ there is a subtle tetragonal$\rightarrow$monoclinic transition associated with emergence of a set of non-magnetic superstructure peaks indexable to a $\vec{k}_\mathrm{struc}\approx0.74\vec{c^*}$ incommensurate modulation parallel to the $α$-MnO$_2$ tunnels. Our findings are consistent with a relation previously reported in titanate hollandites, that $x\approx2|\vec{k}_\mathrm{struc}|$. Magnetic Bragg peaks emerge below $T_2=54.5$\,K, and their positions indicate an incommensurate modulated magnetic structure. The model consistent with the data is a dual-$\vec{k}_\mathrm{mag}$ structure with a ferromagnetic $|\vec{k}_\mathrm{mag}|=0$ component and an incommensurate $\vec{k}_\mathrm{mag}\approx0.37\vec{c^*}$, with the latter most likely to be helical. The period of oscillation of the incommensurate magnetic component is in line with predictions based on a Heisenberg spin Hamiltonian [Mandal \textit{et al}. Phys. Rev. B 90, 104420 (2014)]. Below $T_3=24$\,K, there is a magnetic transition, which gives rise to a different set of magnetic Bragg peaks indicative of a highly complex magnetic structure.

cond-mat.mtrl-sci↗

arXiv2026

An anisotropic functional for two-dimensional material systems

Density function theory is the workhorse of modern electronic structure theory. However, its accuracy in practical calculations is limited by the choice of the exchange-correlation potential. In this respect, two-dimensional materials pose a special challenge, as all these materials and their heterostructures have a crucial similarity. The underlying atomic structures are strongly spatially inhomogeneous, implying that current exchange-correlation functionals, that in almost all cases are isotropic, are ill-prepared for an accurate description. We present an anisotropic screened-exchange potential, that remedies this problem and reproduces the band-gap of 2D materials as well as the piecewise linearity of the total energy with fractional occupation number.

cond-mat.mtrl-sci↗

arXiv2026

Thermally-driven reorientation of the Néel vector in altermagnetic MnTe

Altermagnets are novel magnetic systems that possess a spin-polarized electronic band structure without a net magnetic moment, making them promising for device applications. Hexagonal MnTe, a prototypical altermagnet, arguably exhibits the most properties consistent with theoretical predictions, including an anomalous Hall effect despite a vanishing net magnetization, and altermagnetinduced electronic band splitting. However, fundamental questions remain, including why some effects only appear significantly below the magnetic ordering temperature. Here, we resolve this discrepancy by revealing a reorientation of the Néel vector in single-crystalline MnTe. The Néel vector points 30° from the a-axis at low $T$, before aligning directly with the a-axis around $T\simeq 260$ K. We attribute this to single-ion anisotropy, which depends on temperature-dependent lattice parameters. We obtained these results using muon-spin spectroscopy, magnetization measurements, and X-ray diffraction; we show that the findings are consistent with neutron diffraction. Manipulating this effect, for example through strain, could unlock sensitive electronic detection schemes for external stimuli, paving the way for functional altermagnetic devices.

cond-mat.mtrl-sci↗