20.4.3.3.11 Large Language Models for Autonomous Driving, LLM, LVLM

Chapter Contents (Back)
Large Language Models. LLM. Visual Reasoning. Driver Models. Driver Behavior. Autonomous Vehicles.
See also Vision-Language Navigation.
See also Driver Modeling, Behavior Models, Analysis.
See also General Spatial Reasoning and Geometric Reasoning Issues, Visual Relations.
See also Foundation Models, Graph Foundation Models.

Lin, B.Q.[Bing-Qian], Nie, Y.[Yunshuang], Wei, Z.M.[Zi-Ming], Chen, J.Q.[Jia-Qi], Ma, S.[Shikui], Han, J.H.[Jian-Hua], Xu, H.[Hang], Chang, X.J.[Xiao-Jun], Liang, X.D.[Xiao-Dan],
NavCoT: Boosting LLM-Based Vision-and-Language Navigation via Learning Disentangled Reasoning,
PAMI(47), No. 7, July 2025, pp. 5945-5957.
IEEE DOI 2506
Navigation, Cognition, Training, Scalability, Glass, Planning, Large language models, History, Visualization, disentangled reasoning BibRef

Ding, X.P.[Xin-Peng], Han, J.H.[Jian-Hua], Xu, H.[Hang], Zhang, W.[Wei], Li, X.M.[Xiao-Meng],
HiLM-D: Enhancing MLLMs with Multi-scale High-Resolution Details for Autonomous Driving,
IJCV(133), No. 8, August 2025, pp. 5379-5395.
Springer DOI 2508
BibRef

Ding, X.P.[Xin-Peng], Han, J.H.[Jian-Hua], Xu, H.[Hang], Liang, X.D.[Xiao-Dan], Zhang, W.[Wei], Li, X.M.[Xiao-Meng],
Holistic Autonomous Driving Understanding by Bird'View Injected Multi-Modal Large Models,
CVPR24(13668-13677)
IEEE DOI Code:
WWW Link. 2410
Bridges, Large language models, Semantics, Autonomous vehicles BibRef

Liu, T.Q.[Tian-Qi], Qin, Y.J.[Yan-Jun], Zhang, S.H.[Shang-Hang], Tao, X.M.[Xiao-Ming],
Empowering Corner Case Detection in Autonomous Vehicles With Multimodal Large Language Models,
SPLetters(32), 2025, pp. 51-55.
IEEE DOI 2501
Rare objects in odd locations. Object detection, Visualization, Autonomous vehicles, Large language models, Roads, Vectors, Transformers, object detection BibRef

Wu, M.Y.[Meng-Yao], Yu, F.R.[F. Richard], Liu, P.X.P.[Peter Xiao-Ping], He, Y.[Ying],
Facilitating Autonomous Driving Tasks With Large Language Models,
IEEE_Int_Sys(40), No. 1, January 2025, pp. 45-52.
IEEE DOI 2502
Safety, Decision making, Autonomous vehicles, Reinforcement learning, Statistical learning, Autonomous driving BibRef

Cao, J.H.[Jing-Hao], Liu, S.[Sheng], Wu, C.F.[Chao-Fan], Li, Y.[Yang], Du, S.[Sidan],
ATHENA - Autonomous Vehicle Trajectory Planning Considered Human Action Awareness,
SPLetters(32), 2025, pp. 1845-1849.
IEEE DOI 2505
Pedestrians, Trajectory planning, Autonomous vehicles, Trajectory, Prompt engineering, Training, Vectors, Large language models, multi-agent BibRef


Chen, K.[Kai], Li, Y.Z.[Yan-Ze], Zhang, W.H.[Wen-Hua], Liu, Y.X.[Yan-Xin], Li, P.X.[Peng-Xiang], Gao, R.[Ruiyuan], Hong, L.Q.[Lan-Qing], Tian, M.[Meng], Zhao, X.H.[Xin-Hai], Li, Z.G.[Zhen-Guo], Yeung, D.Y.[Dit-Yan], Lu, H.C.[Hu-Chuan], Jia, X.[Xu],
Automated Evaluation of Large Vision-Language Models on Self-Driving Corner Cases,
WACV25(7817-7826)
IEEE DOI 2505
Roads, Catalysts, Large language models, Benchmark testing, Data structures, Reliability, Autonomous vehicles, interpretable autonomous agents BibRef

Renz, K.[Katrin], Chen, L.[Long], Arani, E.[Elahe], Sinavski, O.[Oleg],
SimLingo: Vision-Only Closed-Loop Autonomous Driving with Language-Action Alignment,
CVPR25(11993-12003)
IEEE DOI Code:
WWW Link. 2508
Visualization, Laser radar, Large language models, Benchmark testing, Cameras, VLA BibRef

Zhang, Z.Y.[Zhi-Yuan], Li, X.F.[Xiao-Fan], Xu, Z.H.[Zhi-Hao], Peng, W.J.[Wen-Jie], Zhou, Z.J.[Zi-Jian], Shi, M.J.[Miao-Jing], Huang, S.P.[Shuang-Ping],
MPDrive: Improving Spatial Understanding with Marker-Based Prompt Learning for Autonomous Driving,
CVPR25(12089-12099)
IEEE DOI 2508
Visualization, Accuracy, Semantics, Transforms, Predictive models, Feature extraction, Question answering (information retrieval), multimodal large language model BibRef

Xu, Z.H.[Zhen-Hua], Bai, Y.[Yan], Zhang, Y.J.[Yu-Jia], Li, Z.L.[Zhuo-Ling], Xia, F.[Fei], Wong, K.Y.K.[Kwan-Yee K.], Wang, J.Q.[Jian-Qiang], Zhao, H.S.[Heng-Shuang],
DriveGPT4-V2: Harnessing Large Language Model Capabilities for Enhanced Closed-Loop Autonomous Driving,
CVPR25(17261-17270)
IEEE DOI 2508
Visualization, Large language models, Imitation learning, Process control, Predictive models, Reliability, Videos BibRef

Hegde, D.[Deepti], Yasarla, R.[Rajeev], Cai, H.[Hong], Han, S.Z.[Shi-Zhong], Bhattacharyya, A.[Apratim], Mahajan, S.[Shweta], Liu, L.T.[Li-Tian], Garrepalli, R.[Risheek], Patel, V.M.[Vishal M.], Porikli, F.M.[Fatih M.],
Distilling Multi-Modal Large Language Models for Autonomous Driving,
CVPR25(27575-27585)
IEEE DOI 2508
Training, Heavily-tailed distribution, Navigation, Large language models, Planning, Trajectory, end-to-end planning BibRef

Chen, Y.[Yuan], Ding, Z.H.[Zi-Han], Wang, Z.Q.[Zi-Qin], Wang, Y.[Yan], Zhang, L.J.[Li-Jun], Liu, S.[Si],
Asynchronous Large Language Model Enhanced Planner for Autonomous Driving,
ECCV24(XXXVI: 22-38).
Springer DOI 2412
BibRef

Li, B.[Boyi], Wang, Y.[Yue], Mao, J.[Jiageng], Ivanovic, B.[Boris], Veer, S.[Sushant], Leung, K.[Karen], Pavone, M.[Marco],
Driving Everywhere with Large Language Model Policy Adaptation,
CVPR24(14948-14957)
IEEE DOI 2410
Measurement, Video on demand, Accuracy, Large language models, Planning, Large Language Models, Driving Copilot BibRef

Wei, Y.X.[Yu-Xi], Wang, Z.[Zi], Lu, Y.F.[Yi-Fan], Xu, C.X.[Chen-Xin], Liu, C.X.[Chang-Xing], Zhao, H.[Hao], Chen, S.[Siheng], Wang, Y.F.[Yan-Feng],
Editable Scene Simulation for Autonomous Driving via Collaborative LLM-Agents,
CVPR24(15077-15087)
IEEE DOI Code:
WWW Link. 2410
Large language models, Face recognition, Natural languages, Collaboration, Lighting, Rendering (computer graphics), LLM agent BibRef

Shao, H.[Hao], Hu, Y.X.[Yu-Xuan], Wang, L.[Letian], Song, G.L.[Guang-Lu], Waslander, S.L.[Steven L.], Liu, Y.[Yu], Li, H.S.[Hong-Sheng],
LMDrive: Closed-Loop End-to-End Driving with Large Language Models,
CVPR24(15120-15130)
IEEE DOI 2410
Navigation, Large language models, Multimodal sensors, Natural languages, Benchmark testing, Software, LLM, autonomous driving BibRef

Ma, Y.S.[Yun-Sheng], Cui, C.[Can], Cao, X.[Xu], Ye, W.Q.[Wen-Qian], Liu, P.R.[Pei-Ran], Lu, J.[Juanwu], Abdelraouf, A.[Amr], Gupta, R.[Rohit], Han, K.T.[Kyung-Tae], Bera, A.[Aniket], Rehg, J.M.[James M.], Wang, Z.[Ziran],
LaMPilot: An Open Benchmark Dataset for Autonomous Driving with Language Model Programs,
CVPR24(15141-15151)
IEEE DOI 2410
Codes, Large language models, Benchmark testing, Cognition, Safety, Pattern recognition BibRef

Zhang, J.W.[Jia-Wei], Xu, C.[Chejian], Li, B.[Bo],
ChatScene: Knowledge-Enabled Safety-Critical Scenario Generation for Autonomous Vehicles,
CVPR24(15459-15469)
IEEE DOI Code:
WWW Link. 2410
Training, Codes, Large language models, Transforms, Robustness, Safety, Autonomous Driving, Large Language Model, Safety-Critical Scenario BibRef

Sirnam, S.[Swetha], Yang, J.[Jinyu], Neiman, T.[Tal], Rizve, M.N.[Mamshad Nayeem], Tran, S.[Son], Yao, B.[Benjamin], Chilimbi, T.[Trishul], Shah, M.[Mubarak],
X-former: Unifying Contrastive and Reconstruction Learning for MLLMs,
ECCV24(VI: 146-162).
Springer DOI 2412
BibRef

Qiao, Y.Y.[Yan-Yuan], Liu, Q.Y.[Qian-Yi], Liu, J.J.[Jia-Jun], Liu, J.[Jing], Wu, Q.[Qi],
LLM as Copilot for Coarse-grained Vision-and-language Navigation,
ECCV24(V: 459-476).
Springer DOI 2412
BibRef

Zhang, J.Y.[Jimu-Yang], Huang, Z.M.[Zan-Ming], Ray, A.[Arijit], Ohn-Bar, E.[Eshed],
Feedback-Guided Autonomous Driving,
CVPR24(15000-15011)
IEEE DOI 2410
Training, Large language models, Cloning, Network architecture, Real-time systems, Autonomous Driving, Large Language Model BibRef

Yang, Y.[Yi], Zhang, Q.W.[Qing-Wen], Li, C.[Ci], Marta, D.S.[Daniel Simões], Batool, N.[Nazre], Folkesson, J.[John],
Human-Centric Autonomous Systems With LLMs for User Command Reasoning,
LLVMCrive24(988-994)
IEEE DOI 2404
Codes, Natural languages, Fasteners, Cognition, Reliability BibRef

Cui, C.[Can], Ma, Y.S.[Yun-Sheng], Cao, X.[Xu], Ye, W.Q.[Wen-Qian], Zhou, Y.[Yang], Liang, K.[Kaizhao], Chen, J.[Jintai], Lu, J.[Juanwu], Yang, Z.[Zichong], Liao, K.D.[Kuei-Da], Gao, T.[Tianren], Li, E.[Erlong], Tang, K.[Kun], Cao, Z.P.[Zhi-Peng], Zhou, T.[Tong], Liu, A.[Ao], Yan, X.R.[Xin-Rui], Mei, S.Q.[Shu-Qi], Cao, J.G.[Jian-Guo], Wang, Z.[Ziran], Zheng, C.[Chao],
A Survey on Multimodal Large Language Models for Autonomous Driving,
LLVMCrive24(958-979)
IEEE DOI 2404
Surveys, Industries, Systematics, Transportation, Benchmark testing BibRef

Fu, D.C.[Dao-Cheng], Li, X.[Xin], Wen, L.C.[Li-Cheng], Dou, M.[Min], Cai, P.L.[Pin-Long], Shi, B.[Botian], Qiao, Y.[Yu],
Drive Like a Human: Rethinking Autonomous Driving with Large Language Models,
LLVMCrive24(910-919)
IEEE DOI Code:
WWW Link. 2404
Industries, Buildings, MIMICs, Drives, Cognition, Autonomous vehicles BibRef

Chapter on Implementations and Applications, Databases, QBIC, Video Analysis, Hardware and Software, Inspection continues in
Large Language Models for VQA, Visual Question Answering .


Last update:Sep 27, 2025 at 16:28:57