Instruction Tuning

ICLR 2024arXiv:2310.03693

#2

Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!

Xiangyu Qi, Yi Zeng, Tinghao Xie et al.

966

ICLR 2024arXiv:2306.14565

#3

Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuning

Fuxiao Liu, Kevin Lin, Linjie Li et al.

422

ICML 2024arXiv:2402.04333

#4

LESS: Selecting Influential Data for Targeted Instruction Tuning

Mengzhou Xia, Sadhika Malladi, Suchin Gururangan et al.

instruction tuningdata selectioninfluence estimationgradient similarity search+4

400

ICCV 2025arXiv:2503.01785

#5

Visual-RFT: Visual Reinforcement Fine-Tuning

Ziyu Liu, Zeyi Sun, Yuhang Zang et al.

reinforcement fine-tuningvision-language modelsverifiable reward functionsobject detection+4

357

ICLR 2024arXiv:2309.07875

#6

Safety-Tuned LLaMAs: Lessons From Improving the Safety of Large Language Models that Follow Instructions

Federico Bianchi, Mirac Suzgun, Giuseppe Attanasio et al.

338

ICLR 2024arXiv:2312.15685

#7

What Makes Good Data for Alignment? A Comprehensive Study of Automatic Data Selection in Instruction Tuning

Wei Liu, Weihao Zeng, Keqing He et al.

337

ICLR 2024arXiv:2306.05087

#8

PandaLM: An Automatic Evaluation Benchmark for LLM Instruction Tuning Optimization

Yidong Wang, Zhuohao Yu, Wenjin Yao et al.

336

ICLR 2024arXiv:2312.01552

#9

The Unlocking Spell on Base LLMs: Rethinking Alignment via In-Context Learning

Bill Yuchen Lin, Abhilasha Ravichander, Ximing Lu et al.

274

ICLR 2024arXiv:2308.07124

#10

OctoPack: Instruction Tuning Code Large Language Models

Niklas Muennighoff, Qian Liu, Armel Zebaze et al.

194

ICML 2024arXiv:2404.14367

#11

Preference Fine-Tuning of LLMs Should Leverage Suboptimal, On-Policy Data

Fahim Tajwar, Anikait Singh, Archit Sharma et al.

preference fine-tuninglarge language modelson-policy reinforcement learningcontrastive learning+4

179

ICLR 2024arXiv:2308.06259

#12

Self-Alignment with Instruction Backtranslation

Xian Li, Ping Yu, Chunting Zhou et al.

167

ICCV 2025arXiv:2412.14164

#13

MetaMorph: Multimodal Understanding and Generation via Instruction Tuning

Shengbang Tong, David Fan, Jiachen Zhu et al.

150

ICML 2024arXiv:2402.02207

#14

Safety Fine-Tuning at (Almost) No Cost: A Baseline for Vision Large Language Models

Yongshuo Zong, Ondrej Bohdal, Tingyang Yu et al.

vision-language modelssafety alignmentinstruction fine-tuningjailbreaking attacks+3

123

ICLR 2024arXiv:2309.10105

#15

Understanding Catastrophic Forgetting in Language Models via Implicit Inference

Suhas Kotha, Jacob Springer, Aditi Raghunathan

114

ICLR 2024arXiv:2310.05914

#16

NEFTune: Noisy Embeddings Improve Instruction Finetuning

Neel Jain, Ping-yeh Chiang, Yuxin Wen et al.

113

ICLR 2025arXiv:2410.01257

#17

HelpSteer2-Preference: Complementing Ratings with Preferences

Zhilin Wang, Alexander Bukharin, Olivier Delalleau et al.

reward modelingbradley-terry modelpreference annotationinstruction following alignment+4

112

ICLR 2024arXiv:2308.07074

#18

#InsTag: Instruction Tagging for Analyzing Supervised Fine-tuning of Large Language Models

Keming Lu, Hongyi Yuan, Zheng Yuan et al.

109

ICLR 2024arXiv:2402.14811

#19

Fine-Tuning Enhances Existing Mechanisms: A Case Study on Entity Tracking

Nikhil Prakash, Tamar Shaham, Tal Haklay et al.

99

ICLR 2024arXiv:2311.12786

#20

Mechanistically analyzing the effects of fine-tuning on procedurally defined tasks

Samyak Jain, Robert Kirk, Ekdeep Singh Lubana et al.

97

ICML 2024arXiv:2402.04833

#21

Long Is More for Alignment: A Simple but Tough-to-Beat Baseline for Instruction Fine-Tuning

Hao Zhao, Maksym Andriushchenko, Francesco Croce et al.

instruction fine-tuningdata quality selectionlong response selectionllm alignment+4

88

COLM 2025arXiv:2504.07912

#22

Echo Chamber: RL Post-training Amplifies Behaviors Learned in Pretraining

Rosie Zhao, Alexandru Meterez, Sham M. Kakade et al.

reinforcement learninglanguage modelspost-trainingppo+2

87

ICML 2024arXiv:2402.05119

#23

A Closer Look at the Limitations of Instruction Tuning

Sreyan Ghosh, Chandra Kiran Evuru, Sonal Kumar et al.

instruction tuninglarge language modelsknowledge degradationhallucination mitigation+3

83

ICLR 2025arXiv:2410.12877

#24

Improving Instruction-Following in Language Models through Activation Steering

Alessandro Stolfo, Vidhisha Balachandran, Safoora Yousefi et al.

80

ECCV 2024arXiv:2403.19522

#25

Model Stock: All we need is just a few fine-tuned models

Dong-Hwan Jang, Sangdoo Yun, Dongyoon Han

fine-tuning methodsweight space analysismodel averagingout-of-distribution generalization+3

77

ICLR 2025arXiv:2308.13320

#26

Fine-tuning can cripple your foundation model; preserving features may be the solution

Philip Torr, Puneet Dokania, Jishnu Mukhoti et al.

76

AAAI 2024arXiv:2307.04408

#27

Teaching Large Language Models to Translate with Comparison

Jiali Zeng, Fandong Meng, Yongjing Yin et al.

73

ICLR 2025arXiv:2407.10490

#28

Learning Dynamics of LLM Finetuning

YI REN, Danica Sutherland

learning dynamicsinstruction tuningpreference tuningdirect preference optimization+4

67

ICML 2024arXiv:2309.10019

#29

Long-Tail Learning with Foundation Model: Heavy Fine-Tuning Hurts

Jiang-Xin Shi, Tong Wei, Zhi Zhou et al.

long-tail learningfoundation modelsfine-tuning paradigmclass imbalance+2

66

ICLR 2024arXiv:2306.11270

#30

Evaluating the Zero-shot Robustness of Instruction-tuned Language Models

Jiuding Sun, Chantal Shaib, Byron Wallace

66

ICLR 2024arXiv:2310.12962

#31

An Emulator for Fine-tuning Large Language Models using Small Language Models

Eric Mitchell, Rafael Rafailov, Archit Sharma et al.

66

ICML 2024arXiv:2406.20053

#32

Covert Malicious Finetuning: Challenges in Safeguarding LLM Adaptation

Danny Halawi, Alexander Wei, Eric Wallace et al.

black-box finetuningmodel safetymalicious adaptationevasion detection+4

65

ICML 2024arXiv:2306.03082

#33

InstructZero: Efficient Instruction Optimization for Black-Box Large Language Models

Lichang Chen, Jiuhai Chen, Tom Goldstein et al.

instruction optimizationblack-box language modelssoft prompt generationbayesian optimization+3

59

ICML 2024arXiv:2402.09497

#34

Instruction Tuning for Secure Code Generation

Jingxuan He, Mark Vero, Gabriela Krasnopolska et al.

secure code generationinstruction tuninglanguage modelssecurity fine-tuning+3

56

ICLR 2024arXiv:2310.05910

#35

SALMON: Self-Alignment with Instructable Reward Models

Zhiqing Sun, Yikang Shen, Hongxin Zhang et al.

55

ICLR 2025arXiv:2406.13542

#36

Self-play with Execution Feedback: Improving Instruction-following Capabilities of Large Language Models

Guanting Dong, Keming Lu, Chengpeng Li et al.

53

ICLR 2025arXiv:2412.15287

#37

Inference-Aware Fine-Tuning for Best-of-N Sampling in Large Language Models

Yinlam Chow, Guy Tennenholtz, Izzeddin Gur et al.

inference-time computebest-of-n samplingimitation learningreinforcement learning+4

49

ICLR 2024arXiv:2211.00635

#38

Two-stage LLM Fine-tuning with Less Specialization and More Generalization

Yihan Wang, Si Si, Daliang Li et al.

large language modelsfine-tuningin-context learningformat specialization+4

43

ICLR 2025arXiv:2406.00888

#39

Aligning Language Models with Demonstrated Feedback

Omar Shaikh, Michelle Lam, Joey Hejna et al.

42

ICML 2024arXiv:2409.01658

#40

From Yes-Men to Truth-Tellers: Addressing Sycophancy in Large Language Models with Pinpoint Tuning

Wei Chen, Zhen Huang, Liang Xie et al.

sycophancy mitigationsupervised pinpoint tuningmodule identificationlarge language models+2

42

COLM 2025arXiv:2501.17703

#41

Critique Fine-Tuning: Learning to Critique is More Effective than Learning to Imitate

Yubo Wang, Xiang Yue, Wenhu Chen

ReasoningLarge Language ModelFine-Tuning

42

ICLR 2024arXiv:2401.12902

#42

Facing the Elephant in the Room: Visual Prompt Tuning or Full finetuning?

Cheng Han, Qifan Wang, Yiming Cui et al.

41

ICLR 2024arXiv:2310.05905

#43

TAIL: Task-specific Adapters for Imitation Learning with Large Pretrained Models

Zuxin Liu, Jesse Zhang, Kavosh Asadi et al.

39

ICLR 2024arXiv:2310.19698

#44

When Do Prompting and Prefix-Tuning Work? A Theory of Capabilities and Limitations

Aleksandar Petrov, Philip Torr, Adel Bibi

39

CVPR 2024arXiv:2403.02969

#45

Multi-modal Instruction Tuned LLMs with Fine-grained Visual Perception

Junwen He, Yifan Wang, Lijun Wang et al.

39

NEURIPS 2025arXiv:2507.02833

#46

Generalizing Verifiable Instruction Following

Valentina Pyatkin, Saumya Malik, Victoria Graf et al.

instruction followingoutput constraintsconstraint verificationverifiable rewards+4

38

ICML 2024arXiv:2402.13659

#47

Privacy-Preserving Instructions for Aligning Large Language Models

Da Yu, Peter Kairouz, Sewoong Oh et al.

privacy-preserving alignmentsynthetic instruction generationdifferential privacylarge language models+3

36

ICML 2025arXiv:2501.12895

#48

Test-Time Preference Optimization: On-the-Fly Alignment via Iterative Textual Feedback

Yafu Li, Xuyang Hu, Xiaoye Qu et al.

35

ICLR 2025arXiv:2410.08020

#49

Efficiently Learning at Test-Time: Active Fine-Tuning of LLMs

Jonas Hübotter, Sascha Bongni, Ido Hakimi et al.

34

ICLR 2025arXiv:2412.13337

#50

Unveiling the Secret Recipe: A Guide For Supervised Fine-Tuning Small LLMs

Aldo Pareja, Nikhil Shivakumar Nayak, Hao Wang et al.

supervised fine-tuninginstruction-tuning datasetssmall language modelstraining configurations+4

34

ICCV 2025arXiv:2504.01901

#51

Ross3D: Reconstructive Visual Instruction Tuning with 3D-Awareness

Haochen Wang, Yucheng Zhao, Tiancai Wang et al.

3d scene understandingvisual instruction tuning3d-aware supervisioncross-view reconstruction+4

33

ICLR 2025arXiv:2501.01702

#52

AgentRefine: Enhancing Agent Generalization through Refinement Tuning

Dayuan Fu, Keqing He, Yejie Wang et al.

33

NEURIPS 2025arXiv:2507.18624

#53

Checklists Are Better Than Reward Models For Aligning Language Models

Vijay Viswanathan, Yanchao Sun, Xiang Kong et al.

reinforcement learninginstruction followingchecklist feedbackai judges+3

32

CVPR 2025arXiv:2412.05334

#54

Closed-Loop Supervised Fine-Tuning of Tokenized Traffic Models

Zhejun Zhang, Peter Karkus, Maximilian Igl et al.

32

CVPR 2025arXiv:2503.00513

#55

Inst3D-LMM: Instance-Aware 3D Scene Understanding with Multi-modal Instruction Tuning

Hanxun Yu, Wentong Li, Song Wang et al.

30

safety alignmentneuron detection methodsafety neuronsself-attention layers+3

#56

Understanding and Enhancing Safety Mechanisms of LLMs via Safety-Specific Neuron

Yiran Zhao, Wenxuan Zhang, Yuxi Xie et al.

ICLR 2025

29

ICML 2024arXiv:2401.12200

#57

APT: Adaptive Pruning and Tuning Pretrained Language Models for Efficient Training and Inference

Bowen Zhao, Hannaneh Hajishirzi, Qingqing Cao

parameter-efficient fine-tuningstructured pruninglanguage model compressionadaptive parameter tuning+4

28

NEURIPS 2025arXiv:2505.11423

#58

When Thinking Fails: The Pitfalls of Reasoning for Instruction-Following in LLMs

Xiaomin Li, Zhou Yu, Zhiwei Zhang et al.

instruction-following accuracychain-of-thought reasoningreasoning-enhanced llmsconstraint attention+2

28

ICML 2024arXiv:2402.02868

#59

Small Model Can Self-Correct

Haixia Han, Jiaqing Liang, Jie Shi et al.

Fine-tuning Reinforcement Learning Models is Secretly a Forgetting Mitigation Problem

Maciej Wołczyk, Bartłomiej Cupiał, Mateusz Ostaszewski et al.

reinforcement learningfine-tuningforgetting mitigationknowledge retention+4

26

ICML 2024arXiv:2310.02905

#61

Use Your INSTINCT: INSTruction optimization for LLMs usIng Neural bandits Coupled with Transformers

Xiaoqiang Lin, Zhaoxuan Wu, Zhongxiang Dai et al.

instruction optimizationneural bandit algorithmsbayesian optimizationtransformer representations+4

26

ICLR 2024arXiv:2311.00233

#62

Instructive Decoding: Instruction-Tuned Large Language Models are Self-Refiner from Noisy Instructions

Taehyeon Kim, JOONKEE KIM, Gihun Lee et al.

26

ICLR 2024arXiv:2306.03828

#63

Quick-Tune: Quickly Learning Which Pretrained Model to Finetune and How

Sebastian Pineda Arango, Fabio Ferreira, Arlind Kadra et al.

25

NEURIPS 2025arXiv:2502.04194

#64

The Best Instruction-Tuning Data are Those That Fit

Dylan Zhang, Qirun Dai, Hao Peng

supervised fine-tuninginstruction tuninglarge language modelsdata selection+4

23

CVPR 2024arXiv:2405.00256

#65

ASAM: Boosting Segment Anything Model with Adversarial Tuning

Bo Li, Haoke Xiao, Lv Tang

22

AAAI 2025arXiv:2501.08580

#66

Densely Connected Parameter-Efficient Tuning for Referring Image Segmentation

Jiaqi Huang, Zunnan Xu, Ting Liu et al.

referring image segmentationparameter-efficient tuningmultimodal feature alignmentmisaligned encoders+4

21

ICLR 2025arXiv:2406.19292

#67

From Artificial Needles to Real Haystacks: Improving Retrieval Capabilities in LLMs by Finetuning on Synthetic Data

Zheyang Xiong, Vasilis Papageorgiou, Kangwook Lee et al.

synthetic data finetuninglong-context retrievalkey-value retrievalinformation retrieval capabilities+3

19

ICLR 2025arXiv:2408.14774

#68

Instruct-SkillMix: A Powerful Pipeline for LLM Instruction Tuning

Simran Kaur, Simon Park, Anirudh Goyal et al.

instruction tuningskill extractionllm metacognitionsft data generation+3

19

ICLR 2025arXiv:2407.06189

#69

Video-STaR: Self-Training Enables Video Instruction Tuning with Any Supervision

Orr Zohar, Xiaohan Wang, Yonatan Bitton et al.

19

ICLR 2025arXiv:2411.16525

#70

Fundamental Limits of Prompt Tuning Transformers: Universality, Capacity and Efficiency

Jerry Yao-Chieh Hu, Wei-Po Wang, Ammar Gilani et al.

prompt tuningtransformer universalitycomputational complexityattention mechanism+4

18

ICML 2025arXiv:2502.01819

#71

Score as Action: Fine Tuning Diffusion Generative Models by Continuous-time Reinforcement Learning

Hanyang Zhao, Haoxian Chen, Ji Zhang et al.

18

ICLR 2024arXiv:2309.13192

#72

Towards Green AI in Fine-tuning Large Language Models via Adaptive Backpropagation

Kai Huang, Hanyun Yin, Heng Huang et al.

17

NEURIPS 2025arXiv:2505.16270

#73

Transformer Copilot: Learning from The Mistake Log in LLM Fine-tuning

Jiaru Zou, Yikun Ban, Zihao Li et al.

mistake log trackinglogits rectificationsupervised fine-tuningtransformer copilot framework+3

15

ICLR 2025arXiv:2410.02200

#74

Revisiting Prefix-tuning: Statistical Benefits of Reparameterization among Prompts

Minh Le, Chau Nguyen, Huy Nguyen et al.

prefix-tuningprompt-based fine-tuningparameter reparameterizationmixture of experts+4

14

ICML 2024arXiv:2307.01189

#75

Trainable Transformer in Transformer

Abhishek Panigrahi, Sadhika Malladi, Mengzhou Xia et al.

in-context learningtransformer architectureparameter efficient fine-tuninglanguage modeling+2

14

NEURIPS 2025arXiv:2501.18100

#76

Panacea: Mitigating Harmful Fine-tuning for Large Language Models via Post-fine-tuning Perturbation

Yibo Wang, Tiansheng Huang, Li Shen et al.

14

ICLR 2025arXiv:2410.07113

#77

Personalized Visual Instruction Tuning

Renjie Pi, Jianshu Zhang, Tianyang Han et al.

13

ICLR 2025arXiv:2503.00723

#78

Re-Imagining Multimodal Instruction Tuning: A Representation View

Yiyang Liu, James Liang, Ruixiang Tang et al.

multimodal instruction tuningparameter efficient fine tuninglarge multimodal modelszero shot generalization+3

13

ICML 2024arXiv:2402.10208

#79

Recovering the Pre-Fine-Tuning Weights of Generative Models

Eliahu Horwitz, Jonathan Kahana, Yedid Hoshen

weight recoverygenerative model fine-tuninglow-rank adaptationmodel alignment+3

13

ICML 2024arXiv:2312.11875

#80

Sparse is Enough in Fine-tuning Pre-trained Large Language Models

Weixi Song, Zuchao Li, Lefei Zhang et al.

parameter efficient fine tuningpac-bayesian generalizationloss landscape oscillationsgradient distribution sparsity+3

12

ICML 2024arXiv:2403.02695

#81

Controllable Prompt Tuning For Balancing Group Distributional Robustness

Hoang Phan, Andrew Wilson, Qi Lei

prompt tuningdistributional robustnessspurious correlationgroup distributional robustness+3

12

ICLR 2025arXiv:2406.10288

#82

Robust Test-Time Adaptation for Zero-Shot Prompt Tuning

Ding-Chu Zhang, Zhi Zhou, Yufeng Li

Do as I do (Safely): Mitigating Task-Specific Fine-tuning Risks in Large Language Models

Francisco Eiras, Aleksandar Petrov, Philip Torr et al.

11

AAAI 2025arXiv:2402.11532

#84

Chain-of-Instructions: Compositional Instruction Tuning on Large Language Models

Shirley Anugrah Hayati, Taehee Jung, Tristan Bodding-Long et al.

11

ECCV 2024arXiv:2407.11422

#85

Reflective Instruction Tuning: Mitigating Hallucinations in Large Vision-Language Models

Jinrui Zhang, Teng Wang, Haigang Zhang et al.

vision-language modelshallucination mitigationinstruction tuningrationale learning+4

11

CVPR 2025arXiv:2503.17940

#86

FisherTune: Fisher-Guided Robust Tuning of Vision Foundation Models for Domain Generalized Segmentation

Dong Zhao, Jinlong Li, Shuang Wang et al.

vision foundation modelsdomain generalized segmentationfisher information matrixparameter sensitivity analysis+4

instruction finetuningcontext relianceparametric knowledgemodel hallucinations+3

#87

Context-Parametric Inversion: Why Instruction Finetuning May Not Actually Improve Context Reliance

Sachin Goyal, Christina Baek, Zico Kolter et al.

ICLR 2025

NEURIPS 2025arXiv:2509.15155

#88

Self-Improving Embodied Foundation Models

Seyed Kamyar Seyed Ghasemipour, Ayzaan Wahid, Jonathan Tompson et al.

ICML 2025arXiv:2502.11812

#89

Towards Understanding Fine-Tuning Mechanisms of LLMs via Circuit Analysis

Xu Wang, Yan Hu, Wenyu Du et al.

ICLR 2025arXiv:2502.02705

#90

Rapidly Adapting Policies to the Real-World via Simulation-Guided Fine-Tuning

Patrick Yin, Tyler Westenbroek, Ching-An Cheng et al.

AAAI 2025arXiv:2407.05700

#91

InverseCoder: Self-improving Instruction-Tuned Code LLMs with Inverse-Instruct

Yutong Wu, Di Huang, Wenxuan Shi et al.

instruction tuningcode large language modelsdata augmentationinverse instruction generation+3

CVPR 2025arXiv:2412.11509

#92

Skip Tuning: Pre-trained Vision-Language Models are Effective and Efficient Adapters Themselves

Shihan Wu, Ji Zhang, Pengpeng Zeng et al.

vision-language modelsprompt tuningparameter efficient fine-tuninglayer-wise skipping+4

ICLR 2025arXiv:2411.04425

#93

DELIFT: Data Efficient Language model Instruction Fine-Tuning

Ishika Agarwal, Krishnateja Killamsetty, Lucian Popa et al.

instruction tuningdata selection optimizationpairwise utility metricsubmodular functions+3

CVPR 2024arXiv:2404.00913

#94

LLaMA-Excitor: General Instruction Tuning via Indirect Feature Interaction

Bo Zou, Chao Yang, Yu Qiao et al.

ICLR 2025arXiv:2407.07089

#95

Fine-Tuning Attention Modules Only: Enhancing Weight Disentanglement in Task Arithmetic

Ruochen Jin, Bojian Hou, Jiancong Xiao et al.

ECCV 2024arXiv:2401.04339

#96

Memory-Efficient Fine-Tuning for Quantized Diffusion Model

Hyogon Ryu, Seohyun Lim, Hyunjung Shim

quantized diffusion modelsmemory-efficient fine-tuningmodel quantizationgenerative ai+3

NEURIPS 2025arXiv:2412.03565

#97

INST-IT: Boosting Instance Understanding via Explicit Visual Prompt Instruction Tuning

Wujian Peng, Lingchen Meng, Yitong Chen et al.

COLM 2025arXiv:2507.06187

#98

The Delta Learning Hypothesis: Preference Tuning on Weak Data can Yield Strong Gains

Scott Geng, Hamish Ivison, Chun-Liang Li et al.

Preference tuningLLM post-trainingsynthetic dataweak-to-strong generalization

ICCV 2025arXiv:2503.12897

#99

Federated Continual Instruction Tuning

Haiyang Guo, Fanhu Zeng, Fei Zhu et al.

federated learninginstruction tuninglarge multimodal modelscontinual learning+4

7

#100

Scene Map-based Prompt Tuning for Navigation Instruction Generation

Sheng Fan, Rui Liu, Wenguan Wang et al.

CVPR 2025

7