Statistical inference of linear forms in low-rank data with incomplete observations

Abstract

Many important tasks in data science, such as large-scale recommender systems, can naturally be approached as statistical inferences of linear forms for matrix or tensor data with highly incomplete observations. These problems pose unique challenges due to the delicate bias-variance tradeoff and the unknown uncertainties and correlations of test statistics. In this context, we study the statistical inference of general linear forms in low-rank noisy matrix and tensor completion models. For the matrix completion problem, we develop a comprehensive method to test individual linear forms with precise asymptotics, both marginally and jointly, and use these to control the false discovery rate (FDR) through a data-splitting and symmetric aggregation scheme. We then demonstrate that our findings on uncertainty quantification and correlations in matrix settings can be extended to tensors using both independent and dependent initialization and one-step power iteration. An intriguing gap between statistical and computational aspects is observed in the inference process.

Date of Award	2024
Original language	English
Awarding Institution	The Hong Kong University of Science and Technology
Supervisor	Dong XIA (Supervisor)