Search for the nonresonant and resonant production of a Higgs boson in association with an additional scalar boson in the $γγττ$ final state in proton-proton collisions at $\sqrt{s}$ = 13 TeV

Abstract

The results of a search for the production of two scalar bosons in final states with two photons and two tau leptons are presented. The search considers both nonresonant production of a Higgs boson pair, HH, and resonant production via a new boson X which decays either to HH or to H and a new scalar Y. The analysis uses up to 138 fb$^{-1}$ of proton-proton collision data, recorded between 2016 and 2018 by the CMS experiment at the LHC at a center-of-mass energy of 13 TeV. No evidence for signal is found in the data. For the nonresonant production, the observed (expected) upper limit at 95% confidence level (CL) on the HH production cross section is set at 930 (740) fb, corresponding to 33 (26) times the standard model prediction. At 95% CL, HH production is observed (expected) to be excluded for values of $κ_λ$ outside the range between $-$12 ($-$9.4) and 17 (15). Observed (expected) upper limits at 95% CL for the X $\to$ HH cross section are found to be within 160 to 2200 (200 to 1800) fb, depending on the mass of X. In the X $\to$ Y($ττ$)H($γγ$) search, the observed (expected) upper limits on the product of the production cross section and decay branching fractions vary between 0.059$-$1.2 fb (0.087$-$0.68 fb). For the X $\to$ Y($γγ$)H($ττ$) search the observed (expected) upper limits on the product of the production cross section and Y $\to$ $γγ$ branching fraction vary between 0.69$-$15 fb (0.73$-$8.3 fb) in the low Y mass search, tightening constraints on the next-to-minimal supersymmetric standard model, and between 0.64$-$10 fb (0.70$-$7.6 fb) in the high Y mass search.