Psychophysiology & Behavioural Measurement
Focus areas:
- EEG, eye-tracking, biosignals for analyzing quality perception
- Implicit responses to media and interaction, beyond self-report
- Emotion detection and modeling in media consumption
- Cognitive-affective UX modeling using physiological data
- Adaptive systems that respond to cognitive and emotional state
To better understand how people perceive digital systems and content, we apply psychophysiological and behavioral methods in the study of user experience, emotion, and attention. We use electroencephalography (EEG), eye tracking, heart rate variability, and other biosignals to measure implicit responses to media and interaction. This enables us to go beyond self-report and capture continuous, objective indicators of mental workload, attention, fatigue, and affect. These measures feed our Quality of Experience work, where they are turned into models and standards. Ultimately, we seek to build adaptive systems that respond meaningfully to users’ cognitive and emotional states.
Related projects:
What the body says when the questionnaire does not
People are poor witnesses to their own workload, attention and affect, and they are worse when asked afterwards. We record electroencephalography, electrodermal activity, heart rate, pupil size and gaze alongside the task itself, and read them against what the person reports. Where the two disagree, the disagreement is the finding — that is where a system feels effortless and measurably is not, or the other way round.
Measuring inside the headset
Physiological measurement was built for a quiet room and a still participant, and virtual reality offers neither. Much of this line is method work: detecting affect in real time from a three-dimensional model of emotion inside a virtual environment, assessing workload with EEG under environmental noise and noise-cancellation, using eye tracking to judge text readability where the text hangs in space rather than on a page.
Instruments we build, and instruments we take apart
Two directions, one purpose. We develop and validate rapid instruments where the established ones are too slow or too verbal, such as pictographic scales for assessing affective states. And we take consumer devices apart: the stress score of a commercial smartwatch was reconstructed and traced back to its physiological basis, because a number shown to millions of people should be known to mean something.
Systems that adapt
Measurement earns its cost when a system does something with it. We work on applications that read cognitive and emotional state and change their behaviour accordingly: virtual health assistants that adapt to the person in front of them, and mission-critical training that adjusts to a trainee's physiological state under load. The open question in each case is not whether adaptation is possible but when it helps and when it is merely unsettling.
Selected work
- Real-time affect detection in virtual reality: a technique based on a three-dimensional model of affect and EEG signals
- Working with environmental noise and noise-cancellation: A workload assessment with EEG and subjective measures
- Reconstruction and Physiological Basis of Samsung’s Galaxy Watch Stress Score
- Using Eye Tracking for Assessing User Experience of Text Readability in Virtual Reality
- Development and Validation of Pictographic Scales for Rapid Assessment of Affective States in Virtual Reality
- Don’t Worry be Happy - Using virtual environments to induce emotional states measured by subjective scales and heart rate parameters
- When Simulation Meets Physiology: Adaptive Resilience for Mission-Critical XR in Fast-Jet Training
- Designing Adaptive Virtual Health Assistants Using Cognitive and Emotional State Predictions