CIKM'24 Tutorial: Towards Efficient Temporal Graph Learning: Algorithms, Frameworks, and Tools

Abstract

Temporal graphs capture dynamic node relations via temporal edges, finding extensive utility in wide domains where time-varying patterns are crucial. Temporal Graph Neural Networks (TGNNs) have gained significant attention for their effectiveness in representing temporal graphs. However, TGNNs still face significant efficiency challenges in real-world low-resource settings. First, from a data-efficiency standpoint, training TGNNs requires sufficient temporal edges and data labels, which is problematic in practical scenarios with limited data collection and annotation. Second, from a resource-efficiency perspective, TGNN training and inference are computationally demanding due to complex encoding operations, especially on large-scale temporal graphs. Minimizing resource consumption while preserving effectiveness is essential. Inspired by these efficiency challenges, this tutorial systematically introduces state-of-the-art data-efficient and resource-efficient TGNNs, focusing on algorithms, frameworks, and tools, and discusses promising yet under-explored research directions in efficient temporal graph learning. This tutorial aims to benefit researchers and practitioners in data mining, machine learning, and artificial intelligence.

• Time: 1:45 PM - 5:30 PM, Monday, October 21, 2024
  
• Location: 120C, Boise Centre, Boise, Idaho, USA
  
• Slides: [Download]

Outline

• Part I: Introduction
  • Background and Motivations
  • Problem Definitions and Settings
• Part II: Data-Efficient Temporal Graph Learning
  • Key Challenges of Data-Efficient TGNNs
  • Self-Supervised Temporal Graph Learning
  • Weakly-Supervised Temporal Graph Learning
  • Few-Shot Temporal Graph Learning
• Part III: Resource-Efficient TGNNs
  • Key Challenges of Resource-Efficient TGNNs
  • Efficient Discrete-Time TGNN Frameworks
  • Efficient Continuous-Time TGNN Frameworks
  • Efficient Distributed TGNN Training Frameworks
• Part IV: Open Questions and Challenges
  • Generative Pre-training on Temporal Graphs
  • Distributed Training on Temporal Graphs

Presenters

Ruijie Wang is a Postdoctoral Research Associate at the Department of Computer Science, the University of Illinois at Urbana Champaign. He received his Ph.D. in Computer Science from the University of Illinois at Urbana-Champaign. His research interests lie in deep graph learning algorithms for real-world graphs at scale to understand the underlying dynamic patterns and predict future knowledge. He is also generally interested in machine learning and deep learning on graphs, natural language, and time-series data, with applications on social network analysis, knowledge graph, and dynamic systems. He has published more than 30 papers in refereed international conferences and journals including NeurIPS, WWW, ACL, SIGIR, AAAI, CIKM, SenSys, etc.

Wanyu Zhao is a first-year Ph.D. student in Computer Science at the University of Illinois Urbana-Champaign (UIUC). Her current research focuses on developing efficient and scalable systems for temporal graph learning. With an interest in the intersection of systems and machine learning, she aims to explore novel techniques in constructing efficient AI systems through algorithmic insights and comprehensive systems understanding. She is also interested in the application of machine learning to enhance system performance.

Dachun Sun is a senior Ph.D. student in Computer Science at the University of Illinois at Urbana-Champaign (UIUC). His research focuses on computational social analysis with deep graph learning and large language models. Main topics include social network data mining and multimodal embedding for social data. Additionally, his academic interests extend to natural language processing, knowledge graphs, and diffusion-based methods on graphs. He has a dozen of published papers at renowned international conferences and journals including TPAMI, NeurIPS, WWW, AAAI, SIGIR, and more.

Charith Mendis is an Assistant Professor at the University of Illinois at Urbana-Champaign. Previously, he was a visiting faculty researcher at Google and was instrumental in designing and developing the learned TPU cost model used in production. His research interests are in automating compiler construction and in building high-performance ML systems. He received his Ph.D. and Masters from the Massachusetts Institute of Technology and his B.Sc. from the University of Moratuwa. He recently co-led the DARPA ISAT study on "ML Optimized Compilers for Heterogeneous Architectures (MOCHA)". He is the recipient of an NSF CAREER Award, an IEEE Micro Top Picks honorable mention, the William A. Martin outstanding master's thesis award at MIT, a best student paper award, a best paper award, and the university gold medal for his B.Sc. He has published work at both top programming languages venues such as PLDI and ASPLOS as well as at top machine learning venues such as ICML and NeurIPS.

Tarek Abdelzaher is a Professor and Willett Faculty Scholar at the Department of Computer Science, the University of Illinois at Urbana Champaign. He received his Ph.D. in Computer Science from the University of Michigan in 1999. He has authored/coauthored more than 300 refereed publications in real-time computing, CPS/IoT, distributed systems, intelligent networked sensing, machine learning, and control. He served as Editor-in-Chief of the Journal of Real-Time Systems for 20 years, and as Associate Editor of the IEEE Transactions on Mobile Computing, IEEE Transactions on Parallel and Distributed Systems, IEEE Embedded Systems Letters, the ACM Transaction on Sensor Networks, ACM Transactions on Internet Technology, ACM Transactions on Internet of Things, and the Ad Hoc Networks Journal. He chaired (as Program or General Chair) several conferences in his area including RTAS, RTSS, IPSN, Sensys, DCoSS, ICDCS, Infocom, and ICAC. Abdelzaher's research interests lie broadly in understanding and influencing performance and temporal properties of networked embedded, social, and software systems in the face of increasing complexity, distribution, and degree of interaction with an external physical environment. He is a recipient of the IEEE Outstanding Technical Achievement and Leadership Award in Real-time Systems (2012), the Xerox Award for Faculty Research (2011), as well as several best paper awards. He is a fellow of ACM and IEEE.