Enhanced meta learning for few-shot learning, and recommendation system

Abstract

Meta-learning tries to leverage information from similar learning tasks. In the commonly-used bilevel optimization formulation, the shared parameter is learned in the outer loop by minimizing the average loss over all tasks. However, the converged solution may be compromised in that it only focuses on optimizing on a small subset of tasks. To alleviate this problem, we consider meta-learning as a multi-objective optimization (MOO) problem, in which each task is an objective. However, existing MOO solvers need to access all the objectives’ gradients in each iteration, and cannot scale to the huge number of tasks in typical meta-learning settings. To alleviate this problem, we propose a scalable gradient-based solver with the use of mini-batch. We provide theoretical guarantees on the Pareto optimality or Pareto stationarity of the converged solution. Empirical studies on various machine learning settings demonstrate that the proposed method is efficient, and achieves better performance than the baselines, particularly on improving the performance of the poorly-performing tasks and thus alleviating the compromising phenomenon. Moreover, we introduce a Meta Prompt Learning (MPL) method tailored for online recommendation systems. This method leverages a meta prompt to capture useful information from historical data efficiently. The key contributions of the MPL method include a bi-level optimization strategy to retain essential information, a multi-step gradient descent approximation for solution finding, and a comprehensive regret analysis of the system’s performance. Our experiments on datasets such as Tmall, Taobao, and Avazu demonstrate that MPL outperforms state-of-the-art models with lower memory usage and training time.

Date of Award	2024
Original language	English
Awarding Institution	The Hong Kong University of Science and Technology
Supervisor	James Tin Yau KWOK (Supervisor)