Vision-Language Models

CVPR 2024arXiv:2312.14238

#2

InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks

Zhe Chen, Jiannan Wu, Wenhai Wang et al.

2,295

CVPR 2024arXiv:2312.07533

#3

Scaling Autoregressive Models for Content-Rich Text-to-Image Generation

Xin Li, Jing Yu Koh, Alexander Ku et al.

VILA: On Pre-training for Visual Language Models

Ji Lin, Danny Yin, Wei Ping et al.

701

CVPR 2024arXiv:2312.08914

#5

CogAgent: A Visual Language Model for GUI Agents

Wenyi Hong, Weihan Wang, Qingsong Lv et al.

629

CVPR 2024arXiv:2401.06209

#6

Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMs

Shengbang Tong, Zhuang Liu, Yuexiang Zhai et al.

multimodal llmsvisual representation learningclip-blind pairsvision-language models+4

593

ECCV 2024arXiv:2311.17043

#7

LLaMA-VID: An Image is Worth 2 Tokens in Large Language Models

Yanwei Li, Chengyao Wang, Jiaya Jia

499

CVPR 2024arXiv:2311.16922

#8

Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decoding

Sicong Leng, Hang Zhang, Guanzheng Chen et al.

object hallucinationsvision-language modelscontrastive decodingvisual grounding+2

487

CVPR 2024arXiv:2311.06242

#9

Florence-2: Advancing a Unified Representation for a Variety of Vision Tasks

Bin Xiao, Haiping Wu, Weijian Xu et al.

409

CVPR 2024arXiv:2310.14566

#10

HallusionBench: An Advanced Diagnostic Suite for Entangled Language Hallucination and Visual Illusion in Large Vision-Language Models

Tianrui Guan, Fuxiao Liu, Xiyang Wu et al.

vision-language modelslanguage hallucinationvisual illusionimage-context reasoning+3

392

CVPR 2025arXiv:2412.14171

#11

Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spaces

Jihan Yang, Shusheng Yang, Anjali W. Gupta et al.

371

ECCV 2024arXiv:2403.06764

#12

An Image is Worth 1/2 Tokens After Layer 2: Plug-and-Play Inference Acceleration for Large Vision-Language Models

Liang Chen, Haozhe Zhao, Tianyu Liu et al.

attention mechanismvision-language modelsinference accelerationcomputational efficiency+4

368

CVPR 2024arXiv:2311.08046

#13

Chat-UniVi: Unified Visual Representation Empowers Large Language Models with Image and Video Understanding

Peng Jin, Ryuichi Takanobu, Cai Zhang et al.

unified visual representationvision-language modelmultimodal conversationsdynamic visual tokens+2

364

CVPR 2024arXiv:2312.14135

#14

V?: Guided Visual Search as a Core Mechanism in Multimodal LLMs

Penghao Wu, Saining Xie

visual search mechanismmultimodal llmshigh-resolution imagesvisual querying+3

345

ECCV 2024arXiv:2404.12390

#15

BLINK: Multimodal Large Language Models Can See but Not Perceive

Xingyu Fu, Yushi Hu, Bangzheng Li et al.

333

ICLR 2024arXiv:2311.01378

#16

Vision-Language Foundation Models as Effective Robot Imitators

Xinghang Li, Minghuan Liu, Hanbo Zhang et al.

320

CVPR 2024arXiv:2311.15826

#17

GeoChat: Grounded Large Vision-Language Model for Remote Sensing

Kartik Kuckreja, Muhammad Sohail Danish, Muzammal Naseer et al.

319

CVPR 2024arXiv:2312.17172

#18

Unified-IO 2: Scaling Autoregressive Multimodal Models with Vision Language Audio and Action

Jiasen Lu, Christopher Clark, Sangho Lee et al.

autoregressive multimodal modelsmultimodal tokenizationencoder-decoder transformermultimodal mixture of denoisers+4

280

ICCV 2025arXiv:2503.10615

#19

R1-Onevision: Advancing Generalized Multimodal Reasoning through Cross-Modal Formalization

yi yang, Xiaoxuan He, Hongkun Pan et al.

265

AAAI 2024arXiv:2308.06394

#20

Detecting and Preventing Hallucinations in Large Vision Language Models

Anisha Gunjal, Jihan Yin, Erhan Bas

vision language modelsvisual question answeringhallucination detectionmultimodal datasets+4

264

ICML 2024arXiv:2402.07865

#21

Prismatic VLMs: Investigating the Design Space of Visually-Conditioned Language Models

Siddharth Karamcheti, Suraj Nair, Ashwin Balakrishna et al.

visually-conditioned language modelsimage preprocessingvisual question answeringobject localization+4

258

ICLR 2025arXiv:2408.04840

#22

mPLUG-Owl3: Towards Long Image-Sequence Understanding in Multi-Modal Large Language Models

Jiabo Ye, Haiyang Xu, Haowei Liu et al.

multi-modal large language modelslong image-sequence understandingvision-language integrationhyper attention blocks+3

243

CVPR 2024arXiv:2312.00785

#23

Sequential Modeling Enables Scalable Learning for Large Vision Models

Yutong Bai, Xinyang Geng, Karttikeya Mangalam et al.

sequential modelinglarge vision modelsvisual sentencescross-entropy loss+3

235

ICML 2024arXiv:2403.09631

#24

3D-VLA: A 3D Vision-Language-Action Generative World Model

Haoyu Zhen, Xiaowen Qiu, Peihao Chen et al.

vision-language-action models3d perceptiongenerative world modelembodied foundation models+4

233

CVPR 2024arXiv:2307.06949

#25

HyperDreamBooth: HyperNetworks for Fast Personalization of Text-to-Image Models

Nataniel Ruiz, Yuanzhen Li, Varun Jampani et al.

232

ICLR 2025arXiv:2409.04429

#26

VILA-U: a Unified Foundation Model Integrating Visual Understanding and Generation

Yecheng Wu, Zhuoyang Zhang, Junyu Chen et al.

208

AAAI 2024arXiv:2308.09936

#27

BLIVA: A Simple Multimodal LLM for Better Handling of Text-Rich Visual Questions

Wenbo Hu, Yifan Xu, Yi Li et al.

vision language modelsvisual question answeringmultimodal large language modelstext-rich image understanding+4

192

ICLR 2024arXiv:2309.07915

#28

MMICL: Empowering Vision-language Model with Multi-Modal In-Context Learning

Haozhe Zhao, Zefan Cai, Shuzheng Si et al.

191

CVPR 2024arXiv:2404.05726

#29

MA-LMM: Memory-Augmented Large Multimodal Model for Long-Term Video Understanding

Bo He, Hengduo Li, Young Kyun Jang et al.

188

ECCV 2024arXiv:2403.11703

#30

LLaVA-UHD: an LMM Perceiving any Aspect Ratio and High-Resolution Images

Zonghao Guo, Ruyi Xu, Yuan Yao et al.

174

CVPR 2024arXiv:2403.14003

#31

Multi-Modal Hallucination Control by Visual Information Grounding

Alessandro Favero, Luca Zancato, Matthew Trager et al.

160

CVPR 2025arXiv:2412.04468

#32

NVILA: Efficient Frontier Visual Language Models

Zhijian Liu, Ligeng Zhu, Baifeng Shi et al.

157

CVPR 2024arXiv:2312.00784

#33

ViP-LLaVA: Making Large Multimodal Models Understand Arbitrary Visual Prompts

Mu Cai, Haotian Liu, Siva Mustikovela et al.

155

CVPR 2024arXiv:2312.10032

#34

Osprey: Pixel Understanding with Visual Instruction Tuning

Yuqian Yuan, Wentong Li, Jian liu et al.

visual instruction tuningmask-text instructionpixel-level understandingmask-based region-text dataset+4

149

NEURIPS 2025arXiv:2501.01957

#35

VITA-1.5: Towards GPT-4o Level Real-Time Vision and Speech Interaction

Chaoyou Fu, Haojia Lin, Xiong Wang et al.

multimodal large language modelsvision and speech interactionspeech-to-speech dialoguevisual and speech modalities+3

138

ICLR 2024arXiv:2310.12921

#36

Vision-Language Models are Zero-Shot Reward Models for Reinforcement Learning

Juan Rocamonde, Victoriano Montesinos, Elvis Nava et al.

137

ECCV 2024arXiv:2407.21771

#37

Paying More Attention to Images: A Training-Free Method for Alleviating Hallucination in LVLMs

Shi Liu, Kecheng Zheng, Wei Chen

attention mechanismvision-language modelshallucination reductionimage token enhancement+4

134

ICLR 2024arXiv:2311.03054

#38

AnyText: Multilingual Visual Text Generation and Editing

Yuxiang Tuo, Wangmeng Xiang, Jun-Yan He et al.

125

CVPR 2025arXiv:2412.04467

#39

VisionZip: Longer is Better but Not Necessary in Vision Language Models

Senqiao Yang, Yukang Chen, Zhuotao Tian et al.

123

ICLR 2025arXiv:2410.05160

#40

VLM2Vec: Training Vision-Language Models for Massive Multimodal Embedding Tasks

Ziyan Jiang, Rui Meng, Xinyi Yang et al.

117

ICLR 2025arXiv:2408.15998

#41

Eagle: Exploring The Design Space for Multimodal LLMs with Mixture of Encoders

Min Shi, Fuxiao Liu, Shihao Wang et al.

multimodal large language modelsvision encodersoptical character recognitiondocument analysis+4

116

NEURIPS 2025arXiv:2506.15564

#42

Show-o2: Improved Native Unified Multimodal Models

Jinheng Xie, Zhenheng Yang, Mike Zheng Shou

autoregressive modelingflow matchingmultimodal understandingvideo generation+4

106

ECCV 2024arXiv:2311.16101

#43

How Many Unicorns Are in This Image? A Safety Evaluation Benchmark for Vision LLMs

Haoqin Tu, Chenhang Cui, Zijun Wang et al.

105

CVPR 2025arXiv:2410.02712

#44

LLaVA-Critic: Learning to Evaluate Multimodal Models

Tianyi Xiong, Xiyao Wang, Dong Guo et al.

multimodal model evaluationinstruction-following datasetlmm-as-a-judgepreference learning+4

103

ICLR 2024arXiv:2310.09291

#45

Vision-by-Language for Training-Free Compositional Image Retrieval

Shyamgopal Karthik, Karsten Roth, Massimiliano Mancini et al.

102

ICLR 2025arXiv:2403.03003

#46

Feast Your Eyes: Mixture-of-Resolution Adaptation for Multimodal Large Language Models

Gen Luo, Yiyi Zhou, Yuxin Zhang et al.

multimodal large language modelsimage resolution adaptationvisual recognition granularitymixture-of-resolution adapters+4

102

ICLR 2024arXiv:2303.05657

#47

Tag2Text: Guiding Vision-Language Model via Image Tagging

Xinyu Huang, Youcai Zhang, Jinyu Ma et al.

101

ICLR 2025arXiv:2407.01449

#48

ColPali: Efficient Document Retrieval with Vision Language Models

Manuel Faysse, Hugues Sibille, Tony Wu et al.

document retrievalvision language modelsmulti-vector embeddingsvisual document retrieval+3

94

ECCV 2024arXiv:2312.06109

#49

Vary: Scaling up the Vision Vocabulary for Large Vision-Language Models

Haoran Wei, Lingyu Kong, Jinyue Chen et al.

89

AAAI 2024arXiv:2308.12714

#50

VIGC: Visual Instruction Generation and Correction

Théo Delemazure, Jérôme Lang, Grzegorz Pierczyński

visual instruction generationmultimodal large language modelsinstruction-tuning datavision-language tasks+3

88

ICLR 2025arXiv:2403.20271

#51

Draw-and-Understand: Leveraging Visual Prompts to Enable MLLMs to Comprehend What You Want

Weifeng Lin, Xinyu Wei, Ruichuan An et al.

visual promptingmultimodal large language modelspixel-level understandingmulti-domain dataset+4

87

CVPR 2024arXiv:2311.11860

#52

LION: Empowering Multimodal Large Language Model with Dual-Level Visual Knowledge

Gongwei Chen, Leyang Shen, Rui Shao et al.

84

ICLR 2024arXiv:2310.01018

#53

Controlling Vision-Language Models for Multi-Task Image Restoration

Ziwei Luo, Fredrik K. Gustafsson, Zheng Zhao et al.

84

NEURIPS 2025arXiv:2505.17685

#54

FutureSightDrive: Thinking Visually with Spatio-Temporal CoT for Autonomous Driving

Shuang Zeng, Xinyuan Chang, Mengwei Xie et al.

84

ICLR 2024arXiv:2309.04669

#55

Unified Language-Vision Pretraining in LLM with Dynamic Discrete Visual Tokenization

Yang Jin, Kun Xu, Kun Xu et al.

79

ICLR 2024arXiv:2312.06960

#56

Remote Sensing Vision-Language Foundation Models without Annotations via Ground Remote Alignment

Utkarsh Kumar Mall, Cheng Perng Phoo, Meilin Liu et al.

78

ICLR 2024arXiv:2308.12038

#57

Large Multilingual Models Pivot Zero-Shot Multimodal Learning across Languages

Jinyi Hu, Yuan Yao, Chongyi Wang et al.

77

NEURIPS 2025arXiv:2505.23747

#58

Spatial-MLLM: Boosting MLLM Capabilities in Visual-based Spatial Intelligence

Diankun Wu, Fangfu Liu, Yi-Hsin Hung et al.

77

CVPR 2024arXiv:2311.16714

#59

Embodied Multi-Modal Agent trained by an LLM from a Parallel TextWorld

Yijun Yang, Tianyi Zhou, kanxue Li et al.

76

CVPR 2024arXiv:2312.03052

#60

Visual Program Distillation: Distilling Tools and Programmatic Reasoning into Vision-Language Models

Yushi Hu, Otilia Stretcu, Chun-Ta Lu et al.

76

ICML 2024arXiv:2402.13232

#61

A Touch, Vision, and Language Dataset for Multimodal Alignment

Letian Fu, Gaurav Datta, Huang Huang et al.

multimodal alignmenttactile sensingvision-language modelstouch-vision-language+4

74

CVPR 2024arXiv:2312.08870

#62

VISTA-LLAMA: Reducing Hallucination in Video Language Models via Equal Distance to Visual Tokens

Fan Ma, Xiaojie Jin, Heng Wang et al.

73

CVPR 2024arXiv:2403.02330

#63

RegionGPT: Towards Region Understanding Vision Language Model

Qiushan Guo, Shalini De Mello, Danny Yin et al.

73

AAAI 2024arXiv:2309.16429

#64

Diverse and Aligned Audio-to-Video Generation via Text-to-Video Model Adaptation

Guy Yariv, Itai Gat, Sagie Benaim et al.

72

CVPR 2025arXiv:2412.00493

#65

Video-3D LLM: Learning Position-Aware Video Representation for 3D Scene Understanding

Duo Zheng, Shijia Huang, Liwei Wang

3d scene understandingmultimodal large language modelsposition-aware video representationvideo representation alignment+2

70

CVPR 2024arXiv:2311.15383

#66

Visual Programming for Zero-shot Open-Vocabulary 3D Visual Grounding

Zhihao Yuan, Jinke Ren, Chun-Mei Feng et al.

70

CVPR 2024arXiv:2403.01807

#67

ViewDiff: 3D-Consistent Image Generation with Text-to-Image Models

Lukas Höllein, Aljaž Božič, Norman Müller et al.

3d asset generationtext-to-image modelsdiffusion modelsmulti-view image generation+4

69

ICML 2025arXiv:2502.09838

#68

HealthGPT: A Medical Large Vision-Language Model for Unifying Comprehension and Generation via Heterogeneous Knowledge Adaptation

Tianwei Lin, Wenqiao Zhang, Sijing Li et al.

69

ICLR 2025arXiv:2408.02032

#69

Self-Introspective Decoding: Alleviating Hallucinations for Large Vision-Language Models

Fushuo Huo, Wenchao Xu, Zhong Zhang et al.

hallucination mitigationvision-language modelscontrastive decodingtoken selection strategy+2

68

ICLR 2025arXiv:2410.02762

#70

Interpreting and Editing Vision-Language Representations to Mitigate Hallucinations

Nick Jiang, Anish Kachinthaya, Suzanne Petryk et al.

67

ICML 2024arXiv:2402.02235

#71

Image Fusion via Vision-Language Model

Zixiang Zhao, Lilun Deng, Haowen Bai et al.

image fusionvision-language modelinfrared-visible fusionmedical image fusion+4

67

CVPR 2024arXiv:2312.12730

#72

A Closer Look at the Few-Shot Adaptation of Large Vision-Language Models

Julio Silva-Rodríguez, Sina Hajimiri, Ismail Ben Ayed et al.

few-shot adaptationvision-language modelsefficient transfer learninglinear probing+4

65

CVPR 2024arXiv:2404.07449

#73

Learning to Localize Objects Improves Spatial Reasoning in Visual-LLMs

Kanchana Ranasinghe, Satya Narayan Shukla, Omid Poursaeed et al.

63

CVPR 2024arXiv:2401.12425

#74

The Neglected Tails in Vision-Language Models

Shubham Parashar, Tian Liu, Zhiqiu Lin et al.

61

ICLR 2025arXiv:2503.03321

#75

See What You Are Told: Visual Attention Sink in Large Multimodal Models

Seil Kang, Jinyeong Kim, Junhyeok Kim et al.

attention mechanismvisual attention sinkmultimodal modelsvision-language tasks+4

61

NEURIPS 2025arXiv:2507.04447

#76

DreamVLA: A Vision-Language-Action Model Dreamed with Comprehensive World Knowledge

Wenyao Zhang, Hongsi Liu, Zekun Qi et al.

61

CVPR 2025arXiv:2411.16724

#77

Devils in Middle Layers of Large Vision-Language Models: Interpreting, Detecting and Mitigating Object Hallucinations via Attention Lens

Zhangqi Jiang, Junkai Chen, Beier Zhu et al.

59

ECCV 2024arXiv:2404.07204

#78

BRAVE: Broadening the visual encoding of vision-language models

Oguzhan Fatih Kar, Alessio Tonioni, Petra Poklukar et al.

58

ICLR 2024arXiv:2310.10640

#79

LLM Blueprint: Enabling Text-to-Image Generation with Complex and Detailed Prompts

Hanan Gani, Shariq Bhat, Muzammal Naseer et al.

56

ICLR 2025arXiv:2410.07149

#80

Towards Interpreting Visual Information Processing in Vision-Language Models

Clement Neo, Luke Ong, Philip Torr et al.

56

ECCV 2024arXiv:2403.04593

#81

Embodied Understanding of Driving Scenarios

Yunsong Zhou, Linyan Huang, Qingwen Bu et al.

54

AAAI 2025arXiv:2403.20213

#82

VHM: Versatile and Honest Vision Language Model for Remote Sensing Image Analysis

Chao Pang, Xingxing Weng, Jiang Wu et al.

remote sensing image analysisvision language modelimage-text datasetinstruction dataset+4

54

NEURIPS 2025arXiv:2506.09965

#83

Reinforcing Spatial Reasoning in Vision-Language Models with Interwoven Thinking and Visual Drawing

Junfei Wu, Jian Guan, Kaituo Feng et al.

54

CVPR 2024arXiv:2310.00031

#84

Text-Image Alignment for Diffusion-Based Perception

Neehar Kondapaneni, Markus Marks, Manuel Knott et al.

diffusion modelstext-image alignmentsemantic segmentationdepth estimation+4

53

CVPR 2025arXiv:2501.00289

#85

Dual Diffusion for Unified Image Generation and Understanding

Zijie Li, Henry Li, Yichun Shi et al.

52

CVPR 2024arXiv:2312.06059

#86

CONFORM: Contrast is All You Need for High-Fidelity Text-to-Image Diffusion Models

Tuna Han Salih Meral, Enis Simsar, Federico Tombari et al.

CVPR 2024arXiv:2407.06187

#87

JeDi: Joint-Image Diffusion Models for Finetuning-Free Personalized Text-to-Image Generation

Yu Zeng, Vishal M. Patel, Haochen Wang et al.

personalized text-to-image generationdiffusion modelsfinetuning-free personalizationjoint distribution learning+2

CVPR 2024arXiv:2312.12423

#88

Jack of All Tasks Master of Many: Designing General-Purpose Coarse-to-Fine Vision-Language Model

Shraman Pramanick, Guangxing Han, Rui Hou et al.

CVPR 2024arXiv:2403.17589

#89

Dual Memory Networks: A Versatile Adaptation Approach for Vision-Language Models

Yabin Zhang, Wenjie Zhu, Hui Tang et al.

CVPR 2024arXiv:2404.02132

#90

ViTamin: Designing Scalable Vision Models in the Vision-Language Era

Jieneng Chen, Qihang Yu, Xiaohui Shen et al.

CVPR 2025arXiv:2412.02193

#91

LayoutVLM: Differentiable Optimization of 3D Layout via Vision-Language Models

Fan-Yun Sun, Weiyu Liu, Siyi Gu et al.

vision-language models3d spatial reasoningscene layout representationdifferentiable optimization+3

50

ICLR 2025arXiv:2411.04549

#92

Vision Language Models are In-Context Value Learners

Yecheng Jason Ma, Joey Hejna, Chuyuan Fu et al.

vision language modelstemporal value functionin-context learningtemporal ordering problem+4

ECCV 2024arXiv:2311.17404

#93

VITATECS: A Diagnostic Dataset for Temporal Concept Understanding of Video-Language Models

Shicheng Li, Lei Li, Yi Liu et al.

temporal concept understandingvideo-language modelscounterfactual video descriptionsdiagnostic dataset+2

CVPR 2024arXiv:2312.14233

#94

VCoder: Versatile Vision Encoders for Multimodal Large Language Models

Jitesh Jain, Jianwei Yang, Humphrey Shi

ICLR 2024arXiv:2310.13255

#95

Steve-Eye: Equipping LLM-based Embodied Agents with Visual Perception in Open Worlds

Sipeng Zheng, jiazheng liu, Yicheng Feng et al.

CVPR 2025arXiv:2409.18042

#96

EMOVA: Empowering Language Models to See, Hear and Speak with Vivid Emotions

Kai Chen, Yunhao Gou, Runhui Huang et al.

omni-modal foundation modelsspeech tokenizer designvision-language modelsspeech-language models+4

48

ICLR 2025arXiv:2408.02718

#97

MMIU: Multimodal Multi-image Understanding for Evaluating Large Vision-Language Models

Fanqing Meng, Jin Wang, Chuanhao Li et al.

multimodal multi-image understandingvision-language modelsmulti-image relationshipsspatial understanding+4

48

CVPR 2024arXiv:2310.08255

#98

Leveraging Vision-Language Models for Improving Domain Generalization in Image Classification

Sravanti Addepalli, Ashish Asokan, Lakshay Sharma et al.

vision-language modelsdomain generalizationimage classificationmodel distillation+3

48

NEURIPS 2025arXiv:2504.13180

#99

PerceptionLM: Open-Access Data and Models for Detailed Visual Understanding

Jang Hyun Cho, Andrea Madotto, Effrosyni Mavroudi et al.

vision-language modelssynthetic data generationvideo understandingfine-grained video qa+4

47

ECCV 2024arXiv:2408.10433

#100

CLIP-DPO: Vision-Language Models as a Source of Preference for Fixing Hallucinations in LVLMs

Yassine Ouali, Adrian Bulat, Brais Martinez et al.

47