$\mathbb{Z}_2$ Vortex Crystal Candidate in the Triangular $S=1/2$ Quantum Antiferromagnet

Abstract

The prospect of merging the paradigms of geometric frustration on a triangular lattice and bond anisotropies in the strong spin-orbit coupling limit holds tremendous promise in the ongoing hunt for exotic quantum materials. Here we identify a new candidate system to realize such physics, the organic quantum antiferromagnet (CD$_3$ND$_3$)$_2$NaRuCl$_6$. We report a combination of thermodynamic, magneto-elastic and neutron scattering experiments on single-crystals to determine the phase diagram in axial magnetic fields $\mathbf{H \parallel c}$ and propose a minimal model Hamiltonian. (CD$_3$ND$_3$)$_2$NaRuCl$_6$ displays an ideal triangular arrangement of Ru$^{3+}$ ions adopting the spin-orbital entangled $j_{\rm eff} = 1/2$ state. It hosts residual magnetic order below $T_{\rm N} = 0.23$ K and a highly unusual $H-T$ phase diagram including three different incommensurate states. Spin-waves in the high-field polarized regime are well described by a Heisenberg-like triangular lattice Hamiltonian with a potential sub-leading bond dependent anisotropy term $J_{\pm\pm}$. We discuss possible candidate magnetic structures in the various observed phases and propose two mechanisms that could explain the field-dependent incommensurability, requiring either a small ferromagnetic Kitaev term or a tiny magneto-elastic $J-J'$ isosceles distortion driven by pseudospin-lattice coupling. We argue that the multi-$\mathbf{q}$ ground state in zero magnetic field is a prime candidate for hosting the $\mathbb{Z}_2$ vortex crystal proposed on the triangular Heisenberg-Kitaev model. (CD$_3$ND$_3$)$_2$NaRuCl$_6$ is the first member in an extended family of quantum triangular lattice magnets, providing a new playground to study the interplay of geometric frustration and spin-orbit effects.