Abstract:

[Purpose] This study aims to address the problem of decreased accuracy in human gaze estimation in unconstrained environments caused by large-angle head rotations and complex illumination interference. [Application Context] It targets application scenarios such as human-computer interaction under complex conditions and fatigue driving monitoring, focusing primarily on eye image analysis and visual perception tasks in natural environments. [Method] A dual-branch network integrating Efficient Multi-scale Attention (EMA) and Spatial and Channel Reconstruction Convolution (SCConv) is constructed. Local and global features are extracted through a residual network and an improved dynamic pose-aware Agent Swin Transformer, respectively. SCConv is employed to suppress redundancy, EMA is utilized to enhance multi-scale representation, and head pose priors are incorporated for cross-modal compensation. [Results] On the MPII FaceGaze dataset, the average gaze angular error is reduced to 4.23°; in extreme rotation regions (|Yaw|>15° and |Pitch|>15°), the error is only 5.8°. On the more challenging ETH-XGaze dataset, a promising performance of 4.41° is achieved, demonstrating strong cross-dataset generalization capability. Additionally, the model achieves a single-frame inference latency of only 16.1 ms and an inference frame rate of 62.1 FPS. [Conclusion] This feature fusion method effectively resolves the issues of feature redundancy and scale sensitivity in appearance-based gaze estimation. While maintaining industrial-grade real-time performance, it significantly improves estimation accuracy and robustness under extreme poses and complex scenarios, exhibiting high practical value for engineering applications.

Key words: gaze estimation, feature fusion, EMA module, SCConv module, head pose, robustness