Self-supervised Multi-task Distillation for Few-shot Classification

Few-shot classification has gained significant attention owing to the effectiveness in classifying unseen classes with a few annotated images. Although previous works achieve encouraging classification performance, they heavily rely on one-hot labels during the meta-learning process, which may result in the supervision collapse and limited generalization. To address these challenges, the few-shot classification based on a self-supervised multi-task distillation (SMD) is proposed for mitigating the nuisance arising from one-hot labels. Specifically, SMD formulates multiple auxiliary tasks to enhance the cross entropy classification in a multi-task learning manner, including the self-supervised classification task and the self-distilled classification task. These auxiliary tasks do not rely on one-hot labels in meta-learning, which can effectively enhance generalization performance of the model. Finally, extensive experiment results on two benchmark datasets, i.e., CIFAR-FS and FC-100, demonstrate the superiority and effectiveness of SMD.