It starts with a fine-granular perception of every object — pose, shape, action and intent — and builds up to the map, the interactions, the plan and the simulation that tests it.
Choose a theme to see its projects below.
Showing of 16 · ·
A LiDAR-only mapping model that learns from camera features during training and does without them at inference — useful when the cameras are blinded but the scanner is not. LiDAR intensity carries much of the …
Cyclists signal with their bodies — an arm out, a shoulder check, a shift in balance. bikeActions labels those cues on real recordings so models can learn them. Three of the dataset's authors started on it as students …
Where a detector gives a box, LIF-Net gives a body: seventeen 3D joints per pedestrian and cyclist, fused from LiDAR geometry and camera appearance. A box says a person is there. A pose says which way they face, whether …
Detecting the road users around the vehicle and forecasting where they will go — in one network, without a hand-crafted bounding-box interface in between. MASAR looks backward to look forward: it reconstructs each …
Predicting what others will do and deciding what to do next are the same problem seen twice. PlanTRansformer solves them in one goal-conditioned model. Splitting prediction from planning forces the planner to treat …
Cameras alone lose lane geometry behind parked cars and buses. SatMap aligns a satellite image of the same place to the ego pose and fuses it with the multi-view camera features, so the map decoder can complete what the …
A vision-language model decides what the vehicle should do; a world model predicts how the scene will unfold. WorldVLM joins the two: the VLM's behaviour commands condition the driving world model, so the actions stay …
A benchmark where the pedestrians react to the vehicle, so a planner cannot score well by ignoring them. Most driving benchmarks replay recorded agents: whatever the vehicle does, the pedestrian crosses anyway. HABIT …
Describing a driving scene in words — who is where, who matters, what the vehicle should watch — from fused BEV and camera features. A caption is a cheap, readable probe of what a driving model has actually understood. …
A language-model planner that is grounded in a bird's-eye-view map rather than in raw pixels, and is evaluated in closed loop instead of on logged data. Language models can describe a traffic situation convincingly and …
Find every recording of "a cyclist turning left without signalling" by typing it, instead of scrolling through hours of logged drives. Rare behaviour is what breaks automated vehicles, and rare behaviour is exactly what …
Where generative models actually help in an automated driving stack — and where they only look like they do. Written to give newcomers to the field, including our own students, one place to start. Generative AI (GenAI) …
Drone recordings of Munich intersections, seen from directly above, where every road user and every interaction is visible at once. From the air there are no occlusions and no ego bias: you see the whole negotiation …
Simulated pedestrians usually walk in straight lines. Walk-the-Talk generates their motion from a text description, so a scenario can be written rather than scripted. If the people in your simulator do not behave like …
Predicting the trajectory of other road users, especially vulnerable road users (VRUs), is an important aspect of safety and planning efficiency for autonomous vehicles. With recent advances in Deep-Learning-based …
External Human-Machine Interfaces (eHMI) enable interaction between vehicles and Vulnerable Road Users (VRU), for example, to warn VRUs of the car’s presence. Warning systems should warn of the situation’s urgency, …
Joint project of HM and Hochschule Esslingen with Mercedes-Benz and Spleenlab on large language models for automated driving – from camera and lidar perception to language-based action plans.
Joint project using generative AI to overcome the limits of today's automated driving functions. HM's Intelligent Vehicles Lab works on generative sensor data, scene prediction and foundation models.
BMWE · 2023 – 2026Joint research project on end-to-end automated driving in urban traffic (2023–2026). HM's Intelligent Vehicles Lab developed AI methods to detect pedestrians and cyclists and predict their behaviour.
