PlanTRansformer
Predicting what others will do and deciding what to do next are the same problem seen twice. PlanTRansformer solves them in one goal-conditioned model.
What it does
Splitting prediction from planning forces the planner to treat other road users as fixed obstacles, which they are not — they react. Conditioning one transformer on the ego goal lets the model reason about both sides of the interaction at once.
Abstract
Trajectory prediction and planning are fundamental yet disconnected components in autonomous driving. Prediction models forecast surrounding agent motion under unknown intentions, producing multimodal distributions, while planning assumes known ego objectives and generates deterministic trajectories. This mismatch creates a critical bottleneck: prediction lacks supervision for agent intentions, while planning requires this information. Existing prediction models, despite strong benchmarking performance, often remain disconnected from planning constraints such as collision avoidance and dynamic feasibility. We introduce Plan TRansformer (PTR), a unified Gaussian Mixture Transformer framework integrating goal-conditioned prediction, dynamic feasibility, interaction awareness, and lane-level topology reasoning. A teacher-student training strategy progressively masks surrounding agent commands during training to align with inference conditions where agent intentions are unavailable. PTR achieves 4.3%/3.5% improvement in marginal/joint mAP compared to the baseline Motion Transformer (MTR) and 15.5% planning error reduction at 5s horizon compared to GameFormer. The architecture-agnostic design enables application to diverse Transformer-based prediction models. Project Website: https://github.com/SelzerConst/PlanTRansformer
Result
Citation
@inproceedings{Selzer2026PlanTRansfor,
title = {PlanTRansformer: Unified Prediction and Planning with Goal-conditioned Transformer},
author = {Constantin Selzer and Fabian B. Flohr},
url = {https://github.com/SelzerConst/PlanTRansformer},
year = {2026},
date = {2026-01-01},
booktitle = {2026 IEEE Intelligent Vehicles Symposium (IV)},
organization = {IEEE},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}