Towards A Universal Graph Structural Encoder

Abstract

Recent advancements in large-scale pre-training have shown the potential to learn generalizable representations for downstream tasks. In the graph domain, however, capturing and transferring structural information across different graph domains remains challenging, primarily due to the inherent differences in graph topological patterns across various contexts. For example, a social network's structure is fundamentally different from that of a product co-purchase graph. Additionally, most existing models struggle to capture the rich topological complexity of graph structures, leading to inadequate exploration of the graph embedding space. To address these challenges, we propose GFSE, a universal pre-trained graph encoder designed to capture transferable structural patterns across diverse domains such as the web graph, social networks, and citation networks. GFSE is the first cross-domain graph structural encoder pre-trained with multiple self-supervised learning objectives. Built on a Graph Transformer, GFSE incorporates attention mechanisms informed by graph structural information, enabling it to encode intricate multi-level and fine-grained topological features within complex graph structures. The pre-trained GFSE produces generic and theoretically expressive positional and structural encoding for graphs, which can be seamlessly integrated with various downstream graph feature encoders, including graph neural networks for vectorized features and Large Language Models (LLMs) for text-attributed graphs. Comprehensive experiments on synthetic and real-world datasets demonstrate GFSE's capability to significantly enhance the model's performance while requiring substantially less task-specific fine-tuning.