Relativistic Mott transition and high-order van Hove singularity in twisted double bilayer WSe${}_2$: mean-field and functional renormalization group study

Abstract

Experiments on twisted double bilayer tungsten diselenide have demonstrated that moir'e semiconductors can realize a relativistic Mott transition, i.e., a quantum phase transition from a Dirac semimetal to a correlated insulating state, by twist-angle tuning. In addition, signatures of van Hove singularities were observed in the material's moir'e valence bands, suggesting further potential for the emergence of strongly-correlated states. Based on a continuum model, we provide a detailed analysis of the twist-angle dependence of the system's band structure, focusing on the evolution of the Dirac excitations and the Fermi-surface structure with its Lifshitz transitions across the van Hove fillings. We exhibit that the twist angle can be used to band engineer a high-order van Hove singularity, which can be accessed by gate tuning. We then study the magnetic phase diagram of an effective Hubbard model for twisted double bilayer tungsten diselenide on the effective honeycomb superlattice with tight-binding parameters fitted to the two topmost bands of the continuum model. To that end, we employ a self-consistent Hartree-Fock mean-field approach in real space. Fixing the angle-dependent Hubbard interaction based on the experimental findings, we explore a broad parameter range of twist angle, filling, and temperature. We find a rich variety of magnetic states that we expect to be accessible in future experiments, including, e.g., a non-coplanar spin-density wave with non-zero spin chirality and a half-metallic uniaxial spin-density wave. Finally, we employ a functional renormalization group approach to also study the competition between density-wave and superconducting instabilities. For twist angles of $θ=2.0^\circ, 2.5^\circ$, as well as $θ\approx 3.5^\circ$ -- where the high-order van Hove-singularity is found -- we find clear indications for unconventional superconductivity.