Towards mixed image data augmentation : from optimization criteria to accumulating regularization

Abstract

Data augmentation effectively regularizes deep learning models with less effort than obtaining additional data. Recently, many Mixed Sample Data Augmentation (MSDA) methods have been proposed, mixing two or more images into one augmented image. These mixing strategies generalize networks significantly better than traditional augmentation methods like cropping and flipping. They can improve models’ robustness against corrupted data and capabilities in downstream tasks such as object detection. However, most of these methods still have label mismatching problems, and the improvement criteria must be clarified.

This thesis introduces a solution for the label mismatching problem, an experiment on improvement criteria MSDA, and a compelling image mixing strategy, RandMix. The proposed Semantic Proportional Label Generation (SPLG) procedure generates the accurate label after mixing. SPLG is a generic method applicable to data mixing or wrapping techniques. Second, improvement criteria are proposed with the experiment to investigate their effect. From the three commonly known criteria (Data Diversity, Saliency Information, and Local Smoothness), we experimented and observed that data diversity is more critical to standard MSDA methods. Third, a practical mixing strategy RandMix is proposed. It recursively applies augmentations (MixUp, SaliencyMix, or ResizeMix) with a semantic proportional label generation procedure. RandMix provides significant improvement with negligible computational cost by accumulating the regularization effects. RandMix outperforms other CIFAR and ImageNet classification strategies and, surpasses the popular augmentation strategies on object detection tasks. Moreover, RandMix can combine with other traditional augmentation strategies (such as RandAugment) to obtain the highest model’s robustness against input corruption. Based on these extensive experiments, this thesis shows the effectiveness of applying MSDA techniques with a dedicated application strategy and demonstrates the benefit of MSDA.

Date of Award	2023
Original language	English
Awarding Institution	The Hong Kong University of Science and Technology
Supervisor	Bertram Emil SHI (Supervisor) & Kam Tim WOO (Supervisor)