Molecular design of quinoxaline-based electron acceptors for organic solar cells

Abstract

This thesis addresses three key challenges in gaze estimation and facial expression recognition datasets. Firstly, we tackle the issue of annotation inconsistency in gaze datasets by introducing the Gaze Adaptor Module (GAM) to ensure consistent gaze labels across different datasets. Additionally, we propose the Two-stage Transformer-based Gaze-feature Fusion (TTGF) architecture to enhance the fusion of face and eye features, resulting in improved gaze estimation accuracy. Secondly, we address the problem of imbalanced distribution in facial expression recognition datasets. To mitigate bias during training, we propose a recall-based weighting (RBW) strategy, which adjusts the weighting for expressions and demographic categories (such as gender) using the weighted cross-entropy loss. We also introduce a code-mixing synthesis approach to generate a balanced-synthesized dataset, and companion training with the synthesized dataset (CTSD) to reduce model bias. These techniques significantly improve the fairness and accuracy of facial expression recognition models. Lastly, we tackle the challenge of emotional frame sparsity in video datasets. We propose a novel model that incorporates an extra regression token to integrate features from all frames within each video. This enables effective handling of videos with varying lengths and addresses the issue of sparse emotional reaction frames. Experimental results demonstrate the superiority of our architecture in accurately estimating emotional reaction intensity in videos. By addressing these challenges, this thesis contributes to the advancement of gaze estimation and facial expression recognition.

Date of Award	2024
Original language	English
Awarding Institution	The Hong Kong University of Science and Technology
Supervisor	He YAN (Supervisor)