How Does LGBTQIA+ Identity Affect LLM Behavior? Implications for Requirements Engineering of Mental Health AI Systems

Abstract

Large Language Models are now part of healthcare and mental health support systems, raising concerns regarding fairness toward vulnerable populations, including LGBTQIA+ individuals. However, limited empirical work has investigated how explicit LGBTQIA+ identity disclosure influences LLM-generated responses in mental health contexts. In this study, we extracted 50 real mental health questions from the Counsel Chat repository and constructed three prompt conditions for each question: no identity disclosure, explicit straight identity disclosure, and explicit LGBTQIA+ identity disclosure. We generated and analyzed 450 ChatGPT responses across these conditions using binary coding and comparative analysis. Our findings indicate that LGBTQIA+ identity disclosure did not substantially affect response completeness or supportive guidance. However, responses in the LGBTQIA+-explicit condition presented substantially more identity acknowledgment, contextual expansion, unsupported assumptions, and occasional stereotypical reasoning compared to both other conditions. These results suggest that fairness-related concerns in conversational AI systems may emerge through subtle differences in contextual interpretation and explanatory reasoning rather than through overtly harmful outputs. We discuss implications for fairness requirements and the development of LLM-based mental health support systems.