Latency-driven foveation
Predicting where a viewer will look when a streamed VR frame reaches the headset.
- Discipline
- VR & XR
- Context
- Doctoral research · ACM Multimedia 2026
- Research programme
- 2024–present
- Support
- Research Council of Finland · NSTC Taiwan
LDO combines gaze prediction, stabilization, and adaptive foveal sizing to keep detail aligned with the viewer’s gaze.
5 images · Select a thumbnail to explore. Original project photographs and illustrations.
The work
In remote VR, the image arrives after the gaze sample used to prepare it. That delay can leave the sharpest part of the image behind the viewer’s eyes. Building on our gaze-adaptive foveation work, I developed latency-driven optimization with colleagues at the University of Helsinki, Aalto University, and National Tsing Hua University. The framework predicts gaze, stabilizes small movements, and adapts the high-detail region to uncertainty and latency.
Project contributions & outcomes
- Integrated gaze prediction, hysteresis-based stabilization, and adaptive foveal sizing into the ALVR streaming pipeline.
- Connected per-frame foveation parameters on the server with matching reconstruction parameters on the headset.
- Developed Python analysis and parameter-tuning workflows using synchronized gaze logs, video, and latency measurements.
- Evaluated prediction error and perceptual image quality across six VR titles, alongside a collaborative user study.
Tools & focus
What the study found
- Mean gaze-prediction error fell by 24.04%, and 95th-percentile error by 28.17%, versus delayed gaze across the tested games and added-latency conditions.
- The paper reports gains of approximately 1–4 dB in eye-weighted PSNR and 0.2–0.8 in FovVideoVDP over the D-AADT2 baseline.
- In the 30-participant study, visual quality scored 4.33/5 versus 3.82/5 for EyeNexus; overall quality scored 4.35/5 versus 4.07/5. Both differences were statistically significant.
Objective experiments added 30, 60, or 90 ms to recorded baseline motion-to-photon latency of 60.9–79.4 ms. The user study used two games on Meta Quest Pro at a target 120–130 ms motion-to-photon latency and 50 Mbps. Immersion and interaction differences were not statistically significant. Prediction is not uniformly beneficial at low latency; results depend on the content, hardware, and configuration.
Publication & collaboration
First-author paper with Teemu Kämäräinen, Sam Kondori, Yuan-Chun Sun, Matti Siekkinen, and Cheng-Hsin Hsu. ACM Multimedia 2026, Rio de Janeiro; conference scheduled for 10–14 November 2026.