Uncovering EEG Patterns Consistently Associated with Cybersickness Discomfort Using Deep Learning Interpretability Maps
Uncomfortable sensations similar to motion sickness, called cybersickness, can develop when using Virtual Reality (VR) head-mounted displays. Cybersickness poses a hindrance to greater use of VR technology. Brain activity recorded using electroencephalogram (EEG) can be used to unintrusively detect cybersickness and the discomfort it causes. To intervene for mitigation, machine learning algorithms that can extract meaningful signals related to cybersickness discomfort from the rest of the brain data will be required. In this work, we determined which features in EEG data were most helpful for classifying cybersickness-related discomfort by building a framework with neural networks and interpretability maps. Using brain data from two separate auditory event-related potential (ERP) cybersickness user studies, we extracted which spatio-temporal EEG features (from sensor locations and time steps) were most important for discomfort classification. For the first dataset (n=29), across 120 runs of our framework with three different neural networks over multiple random seeds, the models consistently pointed to scalp locations in left frontal, midline central, and right parietal areas as most helpful in determining if EEG data belonged to someone who was experiencing cybersickness discomfort. Similar locations were also tagged in a second dataset (n=36), showing cross-dataset generalizability of our findings across time, VR stimulus, and participant sample. Early periods in the extracted ERP time windows, approximately between 80 and 260 milliseconds post-stimulus, were marked as important for model classification in both datasets. These results help clarify which tagged features can be used for cybersickness-related (and potentially other types of) discomfort classification with EEG in the future. Project webpage at: https://eeg-discomfort-nnilc.github.io/.