Dynamic Multi-Agent Motion Prediction with Redundancy Reduction and History-Aware Interaction

Introduction and goal of the thesis

Learning robust representations of the driving environment is a highly challenging task in autonomous driving systems. Current multi-agent motion prediction methods heavily rely on deep neural networks to process rich spatio-temporal traffic information. Transformer-based and graph attention network-based approaches have been widely adopted to model the interactions among traffic participants and between participants and static map elements, followed by trajectory decoders for future behavior prediction. However, as the performance of deep learning models often scales with the amount of training data, the collection and annotation of large-scale datasets can be prohibitively expensive. Consequently, self-supervised learning methods have gained increasing attention, as they can generate supervision signals from unlabeled data.

In the domain of motion prediction, the application of self-supervised learning is still in its infancy. This research aims to learn enhanced invariant representations of the road environment for self-supervised pretraining. These representations are expected to facilitate the understanding of interactions within the traffic environment, thereby providing crucial contextual information for motion prediction. However, most existing self-supervised learning methods are derived from the image and natural language processing domains, focusing either on static visual data or pure temporal sequences. In autonomous driving, both time-series traffic participant trajectories and static map data must be considered, making the integration of these heterogeneous data sources a significant challenge. Building upon the redundancy reduction approach proposed in RedMotion, this work seeks to extend the idea to dynamic multi-agent scenarios. Historical trajectories of traffic participants will be incorporated to capture their temporal evolution. Additionally, a HiVT-inspired global spatio-temporal interaction module will be integrated to model the dependencies among agents and between agents and the map. The ultimate goal is to enable accurate multi-agent future trajectory prediction.

Tasks and time schedule

1. Literature research and understanding massage passing method
2. Preparation of map data and traffic participants trajectory data
3. Implementation of neural network of redundancy reduction method
4. Implementation of global interaction based on HiVT
5. Evaluation models with test dataset

Tools

• Argoverse 2 dataset
• Redundancy reduction based on Barlow twins (R. Wagner, 2024, RedMotion: Motion Prediction via Redundancy Reduction)
• Global interaction of traffic participants based on translation- rotation-invariant massage passing method. (Z. Zhou, 2022, HiVT: Hierarchical Vector Transformer for Multi-Agent Motion Prediction)

Requirements

• Programming experience with Python, Pytorch
• Good understanding about deep learning and graph neural network
• Knowledge in prediction model and experience with time series data

Contact person

Yiming Xu (email yiming.xu@ikg.uni-hannover.de, Tel. 511-762-2472)

Institute of Cartography and Geoinformatics, Appelstraße 9a, 30167 Hannover