Towards effective graph neural network design via automated search

Abstract

This thesis presents a systematic exploration of automated Graph Neural Network (GNN) design, introducing four novel frameworks and one emerging direction that fundamentally innovate the design spaces and automation strategies for graph-structured data. Through progressive space innovations, we address the core challenges in automated GNN design, demonstrating how carefully crafted search spaces and automation strategies can lead to more effective graph learning systems. Our first framework, AutoGEL, pioneers a new design space that explicitly incorporates edge feature transformations, expanding beyond traditional node-centric architectures to capture richer graph structural information. Building upon this, MpnnDRF introduces a novel unified design space that bridges Message Passing Neural Networks with Receptive Fields, enabling joint optimization of both message passing functions and their receptive field scopes. We further advance the field with S2PGNN and its extension S2PGNN-plus, which innovate in the pre-training space by automating the discovery of optimal pre-training and fine-tuning strategies, significantly improving model transferability and generalization. Looking forward, we propose KALLM-Auto as an emerging direction, exploring how knowledge-augmented language models can enable universal automated machine learning. Across these frameworks, we develop effective optimization strategies to handle the discrete nature of architectural choices and the complexity of bi-level optimization. Comprehensive evaluation across 35 diverse benchmark datasets, spanning node-level, link-level, and graph-level tasks, demonstrates the effectiveness of our space innovations. This work establishes new paradigms for automated graph learning and opens promising directions for future research in automation strategies.

Date of Award	2024
Original language	English
Awarding Institution	The Hong Kong University of Science and Technology
Supervisor	Can YANG (Supervisor) & Wei WANG (Supervisor)