arXiv subjects

Ralf Engbert

Publications and source records attributed to Ralf Engbert.

At least 19 recordsLinked to original sources

arXiv2023

Distinctive pupil and microsaccade-rate signatures in self-recognition

Pupil dynamics and fixational eye movements are primarily involuntary processes that actively support visual perception during fixations. Both measures are known to be sensitive to ongoing cognitive and affective processing. In a visual fixation experiment (N=116) we demonstrate that self-recognition, familiar faces, and unfamiliar faces elicit specific responses in pupil dynamics and microsaccade rate. First, the pupil response comprises an immediate pupil constriction followed by a dilation in response to stimulus onsets. We observe attenuated constriction and greater dilation when faces are recognized compared to unknown faces. This effect is strongest for one's own face. Second, microsaccade rates, which generally show inhibitory responses to incoming stimuli, generate stronger inhibition for familiar faces compared to unknown faces. Again, the strongest inhibition is observed in response to one's own face. Our results imply that eye-related physiological measures expose hidden knowledge in face memory and could contribute to biometric authentication and identity validation procedures.

q-bio.NC↗

arXiv2023

Bayesian Dynamical Modeling of Fixational Eye Movements

Humans constantly move their eyes, even during visual fixations, where miniature (or fixational) eye movements occur involuntarily. Fixational eye movements comprise slow components (physiological drift and tremor) and fast components (microsaccades). The complex dynamics of physiological drift can be modeled qualitatively as a statistically self-avoiding random walk (SAW model, Engbert, Mergenthaler, Sinn, & Pikovsky, 2011). In this study, we implement a data assimilation approach for the SAW model to explain statistics of fixational eye movements and microsaccades in experimental data obtained from high-resolution eye-tracking. We discuss and analyze the likelihood function for the SAW model, which allows us to apply Bayesian parameter estimation at the level of individual human observers. Based on model fitting, we find a relationship between the activation predicted by the SAW model and the occurrence of microsaccades. The model's latent activation relative to microsaccade onsets and offsets using experimental data lends support to the existence of a triggering mechanism for microsaccades. Our findings suggest that the SAW model can capture individual differences and serve as a tool for exploring the relationship between physiological drift and microsaccades as the two most essential components of fixational eye movements. Our results contribute to understanding individual variability in microsaccade behaviors and the role of fixational eye movements in visual information processing.

stat.AP↗

arXiv2023

SEAM: An Integrated Activation-Coupled Model of Sentence Processing and Eye Movements in Reading

Models of eye-movement control during reading, developed largely within psychology, usually focus on visual, attentional, lexical, and motor processes but neglect post-lexical language processing; by contrast, models of sentence comprehension processes, developed largely within psycholinguistics, generally focus only on post-lexical language processes. We present a model that combines these two research threads, by integrating eye-movement control and sentence processing. Developing such an integrated model is extremely challenging and computationally demanding, but such an integration is an important step toward complete mathematical models of natural language comprehension in reading. We combine the SWIFT model of eye-movement control (Seelig et al., 2020, doi:10.1016/j.jmp.2019.102313) with key components of the Lewis and Vasishth sentence processing model (Lewis & Vasishth, 2005, doi:10.1207/s15516709cog0000_25). This integration becomes possible, for the first time, due in part to recent advances in successful parameter identification in dynamical models, which allows us to investigate profile log-likelihoods for individual model parameters. We present a fully implemented proof-of-concept model demonstrating how such an integrated model can be achieved; our approach includes Bayesian model inference with Markov Chain Monte Carlo (MCMC) sampling as a key computational tool. The integrated Sentence-Processing and Eye-Movement Activation-Coupled Model (SEAM) can successfully reproduce eye movement patterns that arise due to similarity-based interference in reading. To our knowledge, this is the first-ever integration of a complete process model of eye-movement control with linguistic dependency completion processes in sentence comprehension. In future work, this proof of concept model will need to be evaluated using a comprehensive set of benchmark data.

q-bio.NC↗

arXiv2021

A dynamical scan path model for task-dependence during scene viewing

In real-world scene perception human observers generate sequences of fixations to move image patches into the high-acuity center of the visual field. Models of visual attention developed over the last 25 years aim to predict two-dimensional probabilities of gaze positions for a given image via saliency maps. Recently, progress has been made on models for the generation of scan paths under the constraints of saliency as well as attentional and oculomotor restrictions. Experimental research demonstrated that task constraints can have a strong impact on viewing behavior. Here we propose a scan path model for both fixation positions and fixation durations, which includes influences of task instructions and interindividual differences. Based on an eye-movement experiment with four different task conditions, we estimated model parameters for each individual observer and task condition using a fully Bayesian dynamical modeling framework using a joint spatial-temporal likelihood approach with sequential estimation. Resulting parameter values demonstrate that model properties such as the attentional span are adjusted to task requirements. Posterior predictive checks indicate that our dynamical model can reproduce task differences in scan path statistics across individual observers.

q-bio.NC↗

arXiv2020

A Mathematical Model of Local and Global Attention in Natural Scene Viewing

Understanding the decision process underlying gaze control is an important question in cognitive neuroscience with applications in diverse fields ranging from psychology to computer vision. The decision for choosing an upcoming saccade target can be framed as a selection process between two states: Should the observer further inspect the information near the current gaze position (local attention) or continue with exploration of other patches of the given scene (global attention)? Here we propose and investigate a mathematical model motivated by switching between these two attentional states during scene viewing. The model is derived from a minimal set of assumptions that generates realistic eye movement behavior. We implemented a Bayesian approach for model parameter inference based on the model's likelihood function. In order to simplify the inference, we applied data augmentation methods that allowed the use of conjugate priors and the construction of an efficient Gibbs sampler. This approach turned out to be numerically efficient and permitted fitting interindividual differences in saccade statistics. Thus, the main contribution of our modeling approach is two--fold; first, we propose a new model for saccade generation in scene viewing. Second, we demonstrate the use of novel methods from Bayesian inference in the field of scan path modeling.

q-bio.NC↗

arXiv2020

Discriminative Viewer Identification using Generative Models of Eye Gaze

We study the problem of identifying viewers of arbitrary images based on their eye gaze. Psychological research has derived generative stochastic models of eye movements. In order to exploit this background knowledge within a discriminatively trained classification model, we derive Fisher kernels from different generative models of eye gaze. Experimentally, we find that the performance of the classifier strongly depends on the underlying generative model. Using an SVM with Fisher kernel improves the classification performance over the underlying generative model.

cs.LG↗

arXiv2019

Task-dependence in scene perception: Head unrestrained viewing using mobile eye-tracking

Real-world scene perception is typically studied in the laboratory using static picture viewing with restrained head position. Consequently, the transfer of results obtained in this paradigm to real-word scenarios has been questioned. The advancement of mobile eye-trackers and the progress in image processing, however, permit a more natural experimental setup that, at the same time, maintains the high experimental control from the standard laboratory setting. We investigated eye movements while participants were standing in front of a projector screen and explored images under four specific task instructions. Eye movements were recorded with a mobile eye-tracking device and raw gaze data was transformed from head-centered into image-centered coordinates. We observed differences between tasks in temporal and spatial eye-movement parameters and found that the bias to fixate images near the center differed between tasks. Our results demonstrate that current mobile eye-tracking technology and a highly controlled design support the study of fine-scaled task dependencies in an experimental setting that permits more natural viewing behavior than the static picture viewing paradigm.

q-bio.NC↗

arXiv2019

Modulation of oculomotor control during reading of mirrored and inverted texts

The interplay between cognitive and oculomotor processes during reading can be explored when the spatial layout of text deviates from the typical display. In this study, we investigate various eye-movement measures during reading of text with experimentally manipulated layout (word-wise and letter-wise mirrored-reversed text as well as inverted and scrambled text). While typical findings (e.g., longer mean fixation times, shorter mean saccades lengths) in reading manipulated texts compared to normal texts were reported in earlier work, little is known about changes of oculomotor targeting observed in within-word landing positions under the above text layouts. Here we carry out precise analyses of landing positions and find substantial changes in the so-called launch-site effect in addition to the expected overall slow-down of reading performance. Specifically, during reading of our manipulated text conditions with reversed letter order (against overall reading direction), we find a reduced launch-site effect, while in all other manipulated text conditions, we observe an increased launch-site effect. Our results clearly indicate that the oculomotor system is highly adaptive when confronted with unusual reading conditions.

q-bio.NC↗

arXiv2019

How spatial frequencies and color drive object search in real-world scenes: A new eye-movement corpus

When studying how people search for objects in scenes, the inhomogeneity of the visual field is often ignored. Due to physiological limitations peripheral vision is blurred and mainly uses coarse-grained information (i.e., low spatial frequencies) for selecting saccade targets, whereas high-acuity central vision uses fine-grained information (i.e., high spatial frequencies) for analysis of details. Here we investigated how spatial frequencies and color affect object search in real-world scenes. Using gaze-contingent filters we attenuated high or low frequencies in central or peripheral vision while viewers searched color or grayscale scenes. Results showed that peripheral filters and central high-pass filters hardly affected search accuracy, whereas accuracy dropped drastically with central low-pass filters. Peripheral filtering increased the time to localize the target by decreasing saccade amplitudes and increasing number and duration of fixations. The use of coarse-grained information in the periphery was limited to color scenes. Central filtering increased the time to verify target identity instead, especially with low-pass filters. We conclude that peripheral vision is critical for object localization and central vision is critical for object identification. Visual guidance during peripheral object localization is dominated by low-frequency color information, whereas high-frequency information, relatively independent of color, is most important for object identification in central vision.

q-bio.NC↗

arXiv2019

Bayesian parameter estimation for the SWIFT model of eye-movement control during reading

Process-oriented theories of cognition must be evaluated against time-ordered observations. Here we present a representative example for data assimilation of the SWIFT model, a dynamical model of the control of spatial fixation position and fixation duration during reading. First, we develop and test an approximate likelihood function of the model, which is a combination of a pseudo-marginal spatial likelihood and an approximate temporal likelihood function. Second, we use a Bayesian approach to parameter inference using an adapative Markov chain Monte Carlo procedure. Our results indicate that model parameters can be estimated reliably for individual subjects. We conclude that approximative Bayesian inference represents a considerable step forward for the area of eye-movement modeling, where modelling of individual data on the basis of process-based dynamic models has not been possible before.

q-bio.NC↗

arXiv2018

Disentangling top-down vs. bottom-up and low-level vs. high-level influences on eye movements over time

Bottom-up and top-down, as well as low-level and high-level factors influence where we fixate when viewing natural scenes. However, the importance of each of these factors and how they interact remains a matter of debate. Here, we disentangle these factors by analysing their influence over time. For this purpose we develop a saliency model which is based on the internal representation of a recent early spatial vision model to measure the low-level bottom-up factor. To measure the influence of high-level bottom-up features, we use a recent DNN-based saliency model. To account for top-down influences, we evaluate the models on two large datasets with different tasks: first, a memorisation task and, second, a search task. Our results lend support to a separation of visual scene exploration into three phases: The first saccade, an initial guided exploration characterised by a gradual broadening of the fixation density, and an steady state which is reached after roughly 10 fixations. Saccade target selection during the initial exploration and in the steady state are related to similar areas of interest, which are better predicted when including high-level features. In the search dataset, fixation locations are determined predominantly by top-down processes. In contrast, the first fixation follows a different fixation density and contains a strong central fixation bias. Nonetheless, first fixations are guided strongly by image properties and as early as 200 ms after image onset, fixations are better predicted by high-level information. We conclude that any low-level bottom-up factors are mainly limited to the generation of the first saccade. All saccades are better explained when high-level features are considered, and later this high-level bottom-up control can be overruled by top-down influences.

q-bio.NC↗

arXiv2018

Searchers adjust their eye movement dynamics to the target characteristics in natural scenes

When searching a target in a natural scene, both the target's visual properties and similarity to the background influence whether (and how fast) humans are able to find it. However, thus far it has been unclear whether searchers adjust the dynamics of their eye movements (e.g., fixation durations, saccade amplitudes) to the target they search for. In our experiment participants searched natural scenes for six artificial targets with different spatial frequency throughout eight consecutive sessions. High-spatial frequency targets led to smaller saccade amplitudes and shorter fixation durations than low-spatial frequency targets if target identity was known before the trial. If a saccade was programmed in the same direction as the previous saccade (saccadic momentum), fixation durations and successive saccade amplitudes were not influenced by target type. Visual saliency and empirical density at the endpoints of saccadic momentum saccades were comparatively low, indicating that these saccades were less selective. Our results demonstrate that searchers adjust their eye movement dynamics to the search target in a sensible fashion, since low-spatial frequencies are visible farther into the periphery than high-spatial frequencies. Additionally, the saccade direction specificity of our effects suggests a separation of saccades into a default scanning mechanism and a selective, target-dependent mechanism.

q-bio.NC↗

arXiv2017

Spatial statistics for gaze patterns in scene viewing: Effects of repeated viewing

Scene viewing is used to study attentional selection in complex but still controlled environments. One of the main observations on eye movements during scene viewing is the inhomogeneous distribution of fixation locations: While some parts of an image are fixated by almost all observers and are inspected repeatedly by the same observer, other image parts remain unfixated by observers even after long exploration intervals. Here, we apply spatial point process methods to investigate the relationship between pairs of fixations. More precisely, we use the pair correlation function (PCF), a powerful statistical tool, to evaluate dependencies between fixation locations along individual scanpaths. We demonstrate that aggregation of fixation locations within four degrees is stronger than expected from chance. Furthermore, the PCF reveals stronger aggregation of fixations when the same image is presented a second time. We use simulations of a dynamical model to show that a narrower spatial attentional span may explain differences in pair correlations between the first and the second inspection of the same image.

q-bio.NC↗

arXiv2016

The temporal evolution of the central fixation bias in scene viewing

When watching the image of a natural scene on a computer screen, observers initially move their eyes towards the center of the image --- a reliable experimental finding termed central fixation bias. This systematic tendency in eye guidance likely masks attentional selection driven by image properties and top-down cognitive processes. Here we show that the central fixation bias can be reduced by delaying the initial saccade relative to image onset. In four scene-viewing experiments we manipulated observers' initial gaze position and delayed their first saccade by a specific time interval relative to the onset of an image. We analyzed the distance to image center over time and show that the central fixation bias of initial fixations was significantly reduced after delayed saccade onsets. We additionally show that selection of the initial saccade target strongly depended on the first saccade latency. Processes influencing the time course of the central fixation bias were investigated by comparing simulations of several dynamic and statistical models. Model comparisons suggest that the central fixation bias is generated by a default activation as a response to the sudden image onset and that this default activation pattern decreases over time. Our results suggest that it may often be preferable to use a modified version of the scene viewing paradigm that decouples image onset from the start signal for scene exploration and explicitly controls the central fixation bias. In general, the initial fixation location and the latency of the first saccade need to be taken into consideration when investigating eye movements during scene viewing.

q-bio.NC↗

arXiv2016

Influence of initial fixation position in scene viewing

During scene perception our eyes generate complex sequences of fixations. Predictors of fixation locations are bottom-up factors like luminance contrast, top-down factors like viewing instruction, and systematic biases like the tendency to place fixations near the center of an image. However, comparatively little is known about the dynamics of scanpaths after experimental manipulation of specific fixation locations. Here we investigate the influence of initial fixation position on subsequent eye-movement behavior on an image. We presented 64 colored photographs to participants who started their scanpaths from one of two experimentally controlled positions in the right or left part of an image. Additionally, we computed the images' saliency maps and classified them as balanced images or images with high saliency values on either the left or right side of a picture. As a result of the starting point manipulation, we found long transients of mean fixation position and a tendency to overshoot to the image side opposite to the starting position. Possible mechanisms for the generation of this overshoot were investigated using numerical simulations of statistical and dynamical models. We conclude that inhibitory tagging is a viable mechanism for dynamical planning of scanpaths.

q-bio.NC↗

arXiv2016

Likelihood-based Parameter Estimation and Comparison of Dynamical Cognitive Models

Dynamical models of cognition play an increasingly important role in driving theoretical and experimental research in psychology. Therefore, parameter estimation, model analysis and comparison of dynamical models are of essential importance. Here we propose a maximum-likelihood approach for model analysis in a fully dynamical framework that includes time-ordered experimental data. Our methods can be applied to dynamical models for the prediction of discrete behavior (e.g., movement onsets), in particular, we use a dynamical model of saccade generation in scene viewing as a case study for our approach. For this model, the likelihood function can be computed directly by numerical simulation, which enables more efficient parameter estimation including Bayesian inference to obtain reliable estimates and corresponding credible intervals. Using hierarchical models inference is even possible for individual observers. Furthermore, our likelihood approach can be used to compare different models. In our example, the dynamical framework is shown to outperform non-dynamical statistical models. Additionally, the likelihood based evaluation differentiates model variants, which produced indistinguishable predictions on hitherto used statistics. Our results indicate that the likelihood approach is a promising framework for dynamical cognitive models.

stat.ME↗

arXiv2015

Spatial frequency processing in the central and peripheral visual field during scene viewing

Visuospatial attention and gaze control depend on the interaction of foveal and peripheral processing. The foveal and peripheral regions of the visual field are differentially sensitive to parts of the spatial-frequency spectrum. In two experiments, we investigated how the selective attenuation of spatial frequencies in the central or the peripheral visual field affects eye-movement behavior during real-world scene viewing. Gaze-contingent low-pass or high-pass filters with varying filter levels (i.e., cutoff frequencies; Experiment 1) or filter sizes (Experiment 2) were applied. Compared to unfiltered control conditions, mean fixation durations increased most with central high-pass and peripheral low-pass filtering. Increasing filter size prolonged fixation durations with peripheral filtering, but not with central filtering. Increasing filter level prolonged fixation durations with low-pass filtering, but not with high-pass filtering. These effects indicate that fixation durations are not always longer under conditions of increased processing difficulty. Saccade amplitudes largely adapted to processing difficulty: amplitudes increased with central filtering and decreased with peripheral filtering; the effects strengthened with increasing filter size and filter level. In addition, we observed a trade-off between saccade timing and saccadic selection, since saccade amplitudes were modulated when fixation durations were unaffected by the experimental manipulations. We conclude that interactions of perception and gaze control are highly sensitive to experimental manipulations of input images as long as the residual information can still be accessed for gaze control.

q-bio.NC↗

arXiv2015

Fixational eye movements: about a binocular slow control mechanism

Even when we look at stationary objects, involuntarily our eyes perform miniature movements and do not stand perfectly still. Such fixational eye movements (FEM) can be decomposed into at least two components: rapid microsaccades and slow (physiological) drift. Despite the general agreement that microsaccades have a central generating mechanism, the origin of drift is less clear. A direct approach to investigate whether drift is also centrally controlled or merely represents peripheral uncorrelated oculomotor noise is to quantify the statistical dependence between the velocity components of the FEM. Here we investigate the dependence between horizontal and vertical velocity components across the eyes during a visual fixation task with human observers. The results are compared with computer-generated surrogate time series containing only drift or only microsaccades. Our analyses show a binocular dependence between FEM velocity components predominantly due to drift. This result supports the existence of a central generating mechanism that modulates not only microsaccades but also drift and helps to explain the neuronal mechanism generating FEM.

q-bio.NC↗