Delving into Youth Perspectives on In-game Gambling-like Elements: A Proof-of-Concept Study Utilising Large Language Models for Analysing User-Generated Text Data

Abstract

This report documents the development, test, and application of Large Language Models (LLMs) for automated text analysis, with a specific focus on gambling-like elements in digital games, such as lootboxes. The project aimed not only to analyse user opinions and attitudes towards these mechanics, but also to advance methodological research in text analysis. By employing prompting techniques and iterative prompt refinement processes, the study sought to test and improve the accuracy of LLM-based text analysis. The findings indicate that while LLMs can effectively identify relevant patterns and themes on par with human coders, there are still challenges in handling more complex tasks, underscoring the need for ongoing refinement in methodologies. The methodological advancements achieved through this study significantly enhance the application of LLMs in real-world text analysis. The research provides valuable insights into how these models can be better utilized to analyze complex, user-generated content.