In-Context Learning

ICML 2024arXiv:2311.06668

#2

In-context Vectors: Making In Context Learning More Effective and Controllable Through Latent Space Steering

Sheng Liu, Haotian Ye, Lei Xing et al.

in-context learninglatent space steeringlarge language modelsvector arithmetic+4

224

ICLR 2024arXiv:2309.10105

#3

Understanding Catastrophic Forgetting in Language Models via Implicit Inference

Suhas Kotha, Jacob Springer, Aditi Raghunathan

114

ICLR 2024arXiv:2306.01195

#4

Consistency-guided Prompt Learning for Vision-Language Models

Shuvendu Roy, Ali Etemad

93

CVPR 2024arXiv:2403.06495

#5

Toward Generalist Anomaly Detection via In-context Residual Learning with Few-shot Sample Prompts

Jiawen Zhu, Guansong Pang

92

ICLR 2024arXiv:2309.17249

#6

Batch Calibration: Rethinking Calibration for In-Context Learning and Prompt Engineering

Han Zhou, Xingchen Wan, Lev Proleev et al.

87

CVPR 2024arXiv:2403.02781

#7

PromptKD: Unsupervised Prompt Distillation for Vision-Language Models

Zheng Li, Xiang Li, xinyi fu et al.

85

ICML 2024arXiv:2401.12973

#8

In-Context Language Learning: Architectures and Algorithms

Ekin Akyürek, Bailin Wang, Yoon Kim et al.

in-context learningformal language learningregular languagestransformer architecture+4

83

ICLR 2024arXiv:2307.12375

#9

In-Context Learning Learns Label Relationships but Is Not Conventional Learning

Jannik Kossen, Yarin Gal, Tom Rainforth

56

CVPR 2024arXiv:2311.13601

#10

Visual In-Context Prompting

Feng Li, Qing Jiang, Hao Zhang et al.

54

CVPR 2024arXiv:2403.08568

#11

Consistent Prompting for Rehearsal-Free Continual Learning

Zhanxin Gao, Jun Cen, Xiaobin Chang

53

ICML 2024arXiv:2402.18819

#12

Dual Operating Modes of In-Context Learning

Ziqian Lin, Kangwook Lee

in-context learningtask retrievaltask learningprobabilistic modeling+3

46

ICLR 2024arXiv:2211.00635

#13

Two-stage LLM Fine-tuning with Less Specialization and More Generalization

Yihan Wang, Si Si, Daliang Li et al.

large language modelsfine-tuningin-context learningformat specialization+4

43

AAAI 2024arXiv:2312.06323

#14

Learning Hierarchical Prompt with Structured Linguistic Knowledge for Vision-Language Models

Yubin Wang, Xinyang Jiang, De Cheng et al.

prompt learningvision-language modelsstructured linguistic knowledgehierarchical prompt tuning+4

42

ICML 2024arXiv:2402.05403

#15

In-Context Principle Learning from Mistakes

Tianjun Zhang, Aman Madaan, Luyu Gao et al.

in-context learningfew-shot promptingmulti-hop question answeringtextual question answering+4

40

ICLR 2024arXiv:2310.19698

#16

When Do Prompting and Prefix-Tuning Work? A Theory of Capabilities and Limitations

Aleksandar Petrov, Philip Torr, Adel Bibi

39

ECCV 2024arXiv:2407.01400

#17

GalLop: Learning global and local prompts for vision-language models

Marc Lafon, Elias Ramzi, Clément Rambour et al.

prompt learningvision-language modelsfew-shot classificationdomain generalization+4

39

CVPR 2024arXiv:2403.20317

#18

Convolutional Prompting meets Language Models for Continual Learning

Anurag Roy, Riddhiman Moulick, Vinay Verma et al.

37

ICML 2024arXiv:2312.03801

#19

Generalization to New Sequential Decision Making Tasks with In-Context Learning

Sharath Chandra Raparthy, Eric Hambro, Robert Kirk et al.

in-context learningsequential decision makingoffline datasetstransformers+4

37

ICML 2025arXiv:2502.14010

#20

Which Attention Heads Matter for In-Context Learning?

Kayo Yin, Jacob Steinhardt

37

AAAI 2025arXiv:2409.01227

#21

Prompt Compression with Context-Aware Sentence Encoding for Fast and Improved LLM Inference

Barys Liskavets, Maxim Ushakov, Shuvendu Roy et al.

prompt compressioncontext-aware sentence encodingcontrastive learningsentence-level compression+3

33

ICLR 2025arXiv:2501.00070

#22

ICLR: In-Context Learning of Representations

Core Francisco Park, Andrew Lee, Ekdeep Singh Lubana et al.

in-context learningrepresentation organizationconceptual role semanticsgraph tracing task+4

32

ICML 2024arXiv:2402.02382

#23

Revisiting the Power of Prompt for Visual Tuning

Yuzhu Wang, Lechao Cheng, Chaowei Fang et al.

visual prompt tuningprompt initializationself-supervised pretrainingtoken prototypes+4

30

ICLR 2024arXiv:2310.00297

#24

Understanding In-Context Learning from Repetitions

Jianhao (Elliott) Yan, Jin Xu, Chiyu Song et al.

30

ICLR 2024arXiv:2310.10873

#25

IDEAL: Influence-Driven Selective Annotations Empower In-Context Learners in Large Language Models

Shaokun Zhang, Xiaobo Xia, Zhaoqing Wang et al.

28

CVPR 2024arXiv:2312.03703

#26

Skeleton-in-Context: Unified Skeleton Sequence Modeling with In-Context Learning

Xinshun Wang, Zhongbin Fang, Xia Li et al.

28

ICML 2024arXiv:2312.12275

#27

Emergence of In-Context Reinforcement Learning from Noise Distillation

Ilya Zisman, Vladislav Kurenkov, Alexander Nikulin et al.

in-context reinforcement learningnoise-induced curriculumdata acquisitionsynthetic noise injection+2

26

ECCV 2024arXiv:2403.06870

#28

Semantic Residual Prompts for Continual Learning

Martin Menabue, Emanuele Frascaroli, Matteo Boschini et al.

24

ICLR 2025arXiv:2405.19874

#29

Is In-Context Learning Sufficient for Instruction Following in LLMs?

Hao Zhao, Maksym Andriushchenko, francesco croce et al.

in-context learninginstruction followingllm alignmentinstruction fine-tuning+2

22

CVPR 2024arXiv:2311.11178

#30

Active Prompt Learning in Vision Language Models

Jihwan Bang, Sumyeong Ahn, Jae-Gil Lee

22

AAAI 2024arXiv:2401.17390

#31

Customizing Language Model Responses with Contrastive In-Context Learning

Xiang Gao, Kamalika Das

contrastive learninglanguage model alignmentin-context learningintent customization+4

20

ICLR 2025arXiv:2406.11194

#32

In-Context Editing: Learning Knowledge from Self-Induced Distributions

Siyuan Qi, Bangcheng Yang, Kailin Jiang et al.

18

AAAI 2024arXiv:2307.00910

#33

CoPL: Contextual Prompt Learning for Vision-Language Understanding

Koustava Goswami, Srikrishna Karanam, Prateksha Udhayanan et al.

18

ECCV 2024arXiv:2312.03584

#34

Context Diffusion: In-Context Aware Image Generation

Ivona Najdenkoska, Animesh Sinha, Abhimanyu Dubey et al.

in-context learningdiffusion modelsimage generationvisual context encoding+3

17

ICML 2025arXiv:2410.24087

#35

In-Context Fine-Tuning for Time-Series Foundation Models

Matthew Faw, Rajat Sen, Yichen Zhou et al.

16

ICML 2024arXiv:2407.00256

#36

One Prompt is not Enough: Automated Construction of a Mixture-of-Expert Prompts

Ruochen Wang, Sohyun An, Minhao Cheng et al.

instruction design automationmixture-of-expert paradigmin-context learningkernel regression theory+3

16

ICML 2024arXiv:2402.03969

#37

In-Context Learning Agents Are Asymmetric Belief Updaters

Johannes A. Schubert, Akshay Kumar Jagadish, Marcel Binz et al.

in-context learningbelief updatinglarge language modelscognitive psychology tasks+3

16

AAAI 2025arXiv:2412.08176

#38

TextRefiner: Internal Visual Feature as Efficient Refiner for Vision-Language Models Prompt Tuning

Jingjing Xie, Yuxin Zhang, Jun Peng et al.

15

ICLR 2025arXiv:2410.23042

#39

Toward Understanding In-context vs. In-weight Learning

Bryan Chan, Xinyi Chen, Andras Gyorgy et al.

in-context learningin-weight learningtransformer modelsgating mechanism+4

15

ICML 2024arXiv:2310.12274

#40

Learning to Learn Better Visual Prompts

Fengxiang Wang, Wanrong Huang, Shaowu Yang et al.

An Image is Worth Multiple Words: Discovering Object Level Concepts using Multi-Concept Prompt Learning

Chen Jin, Ryutaro Tanno, Amrutha Saseendran et al.

prompt learningtextual inversionmulti-concept learningattention masking+3

13

ICLR 2025arXiv:2410.01280

#42

Sparse Autoencoders Reveal Temporal Difference Learning in Large Language Models

Can Demircan, Tankred Saanum, Akshay Jagadish et al.

12

ICML 2024arXiv:2406.00345

#43

DeCoOp: Robust Prompt Tuning with Out-of-Distribution Detection

Zhi Zhou, Ming Yang, Jiang-Xin Shi et al.

prompt tuningvision-language modelsout-of-distribution detectionzero-shot learning+3

12

AAAI 2025arXiv:2412.08285

#44

Adaptive Prompting for Continual Relation Extraction: A Within-Task Variance Perspective

Minh Le, Tien Ngoc Luu, An Nguyen The et al.

11

ICLR 2025arXiv:2406.00053

#45

Dual Process Learning: Controlling Use of In-Context vs. In-Weights Strategies with Weight Forgetting

Suraj Anand, Michael Lepori, Jack Merullo et al.

11

ICLR 2025arXiv:2405.14660

#46

Implicit In-context Learning

Zhuowei Li, Zihao Xu, Ligong Han et al.

in-context learninglarge language modelsinference-time interventioncontext vector generation+4

10

#47

Context-Parametric Inversion: Why Instruction Finetuning May Not Actually Improve Context Reliance

Sachin Goyal, Christina Baek, Zico Kolter et al.

instruction finetuningcontext relianceparametric knowledgemodel hallucinations+3

10

AAAI 2024arXiv:2401.06799

#48

Make Prompts Adaptable: Bayesian Modeling for Vision-Language Prompt Learning with Data-Dependent Prior

Youngjae Cho, HeeSun Bae, Seungjae Shin et al.

vision-language pretrained modelsprompt learningfew-shot learningbayesian modeling+4

9

ICLR 2024arXiv:2401.14166

#49

BayesPrompt: Prompting Large-Scale Pre-Trained Language Models on Few-shot Inference via Debiased Domain Abstraction

Jiangmeng Li, Fei Song, Yifan Jin et al.

8

ICML 2025arXiv:2505.16694

#50

Beyond Induction Heads: In-Context Meta Learning Induces Multi-Phase Circuit Emergence

Gouki Minegishi, Hiroki Furuta, Shohei Taniguchi et al.

8

COLM 2025arXiv:2504.00132

#51

Contextualize-then-Aggregate: Circuits for In-Context Learning in Gemma-2 2B

Aleksandra Bakalova, Yana Veitsman, Xinting Huang et al.

In-Context LearningMechanistic Interpretability

8

NEURIPS 2025arXiv:2505.00234

#52

Self-Generated In-Context Examples Improve LLM Agents for Sequential Decision-Making Tasks

Vishnu Sarukkai, Zhiqiang Xie, Kayvon Fatahalian

in-context learningsequential decision-makingllm agentstrajectory bootstrapping+3

ICLR 2025arXiv:2503.09032

#53

Teaching LLMs How to Learn with Contextual Fine-Tuning

Younwoo Choi, Muhammad Adil Asif, Ziwen Han et al.

NEURIPS 2025arXiv:2411.02688

#54

On the Loss of Context Awareness in General Instruction Fine-tuning

Yihan Wang, Andrew Bai, Nanyun Peng et al.

#55

Divergence-enhanced Knowledge-guided Context Optimization for Visual-Language Prompt Tuning

Yilun Li, Miaomiao Cheng, Xu Han et al.

prompt tuningvision-language modelsknowledge-guided context optimizationhilbert-schmidt independence criterion+3

ICML 2025arXiv:2410.22316

#56

Understanding Synthetic Context Extension via Retrieval Heads

Xinyu Zhao, Fangcong Yin, Greg Durrett

ICML 2025arXiv:2503.01821

#57

On the Power of Context-Enhanced Learning in LLMs

Xingyu Zhu, Abhishek Panigrahi, Sanjeev Arora

ICML 2024arXiv:2311.12048

#58

One Size Fits All for Semantic Shifts: Adaptive Prompt Tuning for Continual Learning

Doyoung Kim, Susik Yoon, Dongmin Park et al.

continual learningadaptive prompt tuningsemantic shift adaptationprompt management strategies+3

NEURIPS 2025arXiv:2505.17010

#59

Understanding Prompt Tuning and In-Context Learning via Meta-Learning

Tim Genewein, Kevin Li, Jordi Grau-Moya et al.

prompt tuningin-context learningmeta-learningbayesian predictors+4

ICML 2025arXiv:2410.14086

#60

In-Context Learning and Occam's Razor

Eric Elmoznino, Tom Marty, Tejas Kasetty et al.

NEURIPS 2025arXiv:2505.12075

#61

Do different prompting methods yield a common task representation in language models?

Guy Davidson, Todd Gureckis, Brenden Lake et al.

ICML 2024arXiv:2406.01976

#62

Conditional Language Learning with Context

Xiao Zhang, Miao Li, Ji Wu

conditional language modelingselective learningdomain finetuningtopic bias mitigation+3

NEURIPS 2025arXiv:2505.15807

#63

The Atlas of In-Context Learning: How Attention Heads Shape In-Context Retrieval Augmentation

Patrick Kahardipraja, Reduan Achtibat, Thomas Wiegand et al.

attention headsin-context learningretrieval augmentationquestion answering+4

4

AAAI 2025arXiv:2502.14376

#64

A Similarity Paradigm Through Textual Regularization Without Forgetting

Fangming Cui, Jan Fong, Rongfei Zeng et al.

4

ICCV 2025arXiv:2411.13317

#65

Teaching VLMs to Localize Specific Objects from In-context Examples

Sivan Doveh, Nimrod Shabtay, Eli Schwartz et al.

vision-language modelsfew-shot localizationpersonalized object localizationin-context learning+4

3

NEURIPS 2025arXiv:2510.16670

#66

All You Need is One: Capsule Prompt Tuning with a Single Vector

Yiyang Liu, James Liang, Heng Fan et al.

3

ICML 2025arXiv:2506.07440

#67

Federated In-Context Learning: Iterative Refinement for Improved Answer Quality

Ruhan Wang, Zhiyong Wang, Chengkai Huang et al.

federated learningin-context learningquestion answeringlanguage models+4

3

AAAI 2025arXiv:2501.08841

#68

SEC-Prompt:SEmantic Complementary Prompting for Few-Shot Class-Incremental Learning

Ye Liu, Meng Yang

Exploring Task-Level Optimal Prompts for Visual In-Context Learning

Yan Zhu, Huan Ma, Changqing Zhang

visual in-context learningvision foundation modelsprompt search strategiestask-level prompting+1

NEURIPS 2025arXiv:2410.01508

#70

Disentangling Latent Shifts of In-Context Learning with Weak Supervision

Josip Jukić, Jan Šnajder

CVPR 2025arXiv:2504.21263

#71

Embracing Collaboration Over Competition: Condensing Multiple Prompts for Visual In-Context Learning

Jinpeng Wang, Tianci Luo, Yaohua Zha et al.

CVPR 2025arXiv:2506.09473

#72

Provoking Multi-modal Few-Shot LVLM via Exploration-Exploitation In-Context Learning

Cheng Chen, Yunpeng Zhai, Yifan Zhao et al.

in-context learninglarge vision-language modelsmulti-modal demonstration selectionexploration-exploitation reinforcement learning+3

COLM 2025arXiv:2410.20088

#73

RARe: Retrieval Augmented Retrieval with In-Context Examples

Atula Tejaswi, Yoonsang Lee, sujay sanghavi et al.

Information RetrievalIn-Context LearningRepresentation Learning

COLM 2025arXiv:2503.24190

#74

Implicit In-Context Learning: Evidence from Artificial Language Experiments

Xiaomeng Ma, Qihui Xu

implicit learningartificial language learningin-context learningpsycholinguistics+1

ICCV 2025arXiv:2503.07979

#75

Achieving More with Less: Additive Prompt Tuning for Rehearsal-Free Class-Incremental Learning

Haoran Chen, Ping Wang, Zihan Zhou et al.

class-incremental learningparameter-efficient fine-tuningprompt tuningattention computation+2

ICCV 2025arXiv:2509.03803

#76

Causality-guided Prompt Learning for Vision-language Models via Visual Granulation

Mengyu Gao, Qiulei Dong

prompt learningvision-language modelsfine-grained recognitionvisual granulation+4

ICML 2025arXiv:2506.17930

#77

Evolving Prompts In-Context: An Open-ended, Self-replicating Perspective

Jianyu Wang, Zhiqiang Hu, Lidong Bing

#78

RCS-Prompt: Learning Prompt to Rearrange Class Space for Prompt-based Continual Learning

Longrong Yang, Hanbin Zhao, Yunlong Yu et al.

The mechanistic basis of data dependence and abrupt learning in an in-context classification task

Gautam Reddy Nallamala

Chain-of-Focus Prompting: Leveraging Sequential Visual Cues to Prompt Large Autoregressive Vision Models

Jiyang Zheng, Jialiang Shen, Yu Yao et al.

Iterative Vectors: In-Context Gradient Steering without Backpropagation

Yiting Liu, Zhi-Hong Deng

Why In-Context Learning Models are Good Few-Shot Learners?

Shiguang Wu, Yaqing Wang, Quanming Yao

in-context learningfew-shot learningmeta-learningtransformer architecture+4

ICLR 2024arXiv:2309.09888

#83

Evolving Parameterized Prompt Memory for Continual Learning

Muhammad Rifki Kurniawan, Xiang Song, Zhiheng Ma et al.

Context is Environment

Sharut Gupta, Stefanie Jegelka, David Lopez-Paz et al.

ICCV 2025arXiv:2507.22553

#85

X-Prompt: Generalizable Auto-Regressive Visual Learning with In-Context Prompting

Zeyi Sun, Ziyang Chu, Pan Zhang et al.

RainbowPrompt: Diversity-Enhanced Prompt-Evolving for Continual Learning

Kiseong Hong, Gyeong-Hyeon Kim, Eunwoo Kim

prompt-based continual learningclass-incremental learningprompt-evolving mechanismimage classification+3

NEURIPS 2025arXiv:2502.04580

#87

Technical Debt in In-Context Learning: Diminishing Efficiency in Long Context

Taejong Joo, Diego Klabjan

in-context learningtransformer architecturesample complexitybayes optimal estimator+4

in-context learningimage segmentationvisual prompt selectioncontextual example diversity+2

#88

Rethinking and Improving Visual Prompt Selection for In-Context Learning Segmentation Framework

Wei Suo, Lanqing Lai, Mengyang Sun et al.

ECCV 2024

#89

AIM: Let Any Multimodal Large Language Models Embrace Efficient In-Context Learning

Jun Gao, Qian Qiao, Tianxiang Wu et al.

Prompt Gradient Projection for Continual Learning

Jingyang Qiao, Zhizhong Zhang, Xin Tan et al.

Rapid Selection and Ordering of In-Context Demonstrations via Prompt Embedding Clustering

Kha Pham, Hung Le, Man Ngo et al.

Vision and Language Synergy for Rehearsal Free Continual Learning

Muhammad Anwar Masum, Mahardhika Pratama, Savitha Ramasamy et al.

prompt-based approachcontinual learningcatastrophic forgettinglanguage-guided approach+2

COLM 2025arXiv:2507.16217

#93

Towards Compute-Optimal Many-Shot In-Context Learning

Shahriar Golchin, Yanfei Chen, Rujun Han et al.

Many-Shot In-Context LearningIn-Context LearningLarge Language ModelsCompute-Optimal In-Context Learning+1