Analysis of failures in state-of-the-art Person ReID models

Abstract

Humans can reliably find a person through a crowd but computer programs often fail. Even with the help of deep learning, person re-identification (ReID) networks can fail. Studies on reasons for failure have been few. This is because these networks have high dimensional complexity [1]. The lack of understanding limits our ability to improve the ReID networks. This study developed and implemented a real-time person ReID system. The systems were tested to determine the boundary conditions between success and failure. Paths to failure in state-of-the-art deep learning ReID models were analyzed. Findings open up the possibility of future improvements.

We implemented and optimized a real-time ReID system as part of larger screening systems. The systems were tested and deployed at border control points, specifically the Hong Kong International Airport. Test results indicated a discrepancy between the measured accuracy of a model on the training data and on-site performance in real settings. The issue identified was occlusion.

In parallel, we explored how recent state-of-the-art ReID networks decompose and reconstruct the image information and followed the design-of-experiment technique to study and examine the network mechanisms associated with ReID failures. Convergingly, we discovered occlusion also plays a significant part in the failure of the models. Surprisingly, using an occluded query image to search for an occluded match did not improve the performance. Ensuring the query image is not occluded greatly improved model accuracy.

Furthermore, we discovered that retraining using training data that contained occluded samples improved the model accuracy for occluded images but degraded its performance for whole (unoccluded) images. Possible applications of the findings are future enhancements to ReID networks through improvement on the training dataset or through different network architecture designs

Date of Award	2021
Original language	English
Awarding Institution	The Hong Kong University of Science and Technology
Supervisor	Hau Yue Richard SO (Supervisor)