Prompt Engineering

ICLR 2024arXiv:2309.03409

#2

Large Language Models as Optimizers

Chengrun Yang, Xuezhi Wang, Yifeng Lu et al.

683

ICLR 2024arXiv:2310.11324

#3

Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formatting

Melanie Sclar, Yejin Choi, Yulia Tsvetkov et al.

581

ICLR 2024arXiv:2308.03688

#4

AgentBench: Evaluating LLMs as Agents

Xiao Liu, Hao Yu, Hanchen Zhang et al.

543

AAAI 2024arXiv:2308.10144

#5

ExpeL: LLM Agents Are Experiential Learners

Andrew Zhao, Daniel Huang, Quentin Xu et al.

376

ICML 2024arXiv:2309.16797

#6

Promptbreeder: Self-Referential Self-Improvement via Prompt Evolution

Chrisantha Fernando, Dylan Banarse, Henryk Michalewski et al.

prompt evolutionself-referential improvementlarge language modelsreasoning benchmarks+4

364

ICML 2024arXiv:2402.04788

#7

MLLM-as-a-Judge: Assessing Multimodal LLM-as-a-Judge with Vision-Language Benchmark

Dongping Chen, Ruoxi Chen, Shilin Zhang et al.

multimodal large language modelsvision-language benchmarkscoring evaluationpair comparison+4

281

ICLR 2024arXiv:2310.07641

#8

Evaluating Large Language Models at Evaluating Instruction Following

Zhiyuan Zeng, Jiatong Yu, Tianyu Gao et al.

273

ICLR 2024arXiv:2309.10691

#9

MINT: Evaluating LLMs in Multi-turn Interaction with Tools and Language Feedback

Xingyao Wang, Zihan Wang, Jiateng Liu et al.

260

AAAI 2024arXiv:2308.11730

#10

Knowledge Graph Prompting for Multi-Document Question Answering

Yu Wang, Nedim Lipka, Ryan A. Rossi et al.

knowledge graph promptingmulti-document question answeringgraph construction modulegraph traversal module+4

240

ICLR 2025arXiv:2410.02736

#11

Justice or Prejudice? Quantifying Biases in LLM-as-a-Judge

Jiayi Ye, Yanbo Wang, Yue Huang et al.

llm-as-a-judgebias quantificationautomated evaluation frameworklanguage model evaluation+2

229

ICML 2024arXiv:2402.06782

#12

Debating with More Persuasive LLMs Leads to More Truthful Answers

Akbir Khan, John Hughes, Dan Valentine et al.

debate alignmentmodel persuasiontruthful aiexpert oversight+3

212

ICLR 2024arXiv:2310.16427

#13

PromptAgent: Strategic Planning with Language Models Enables Expert-level Prompt Optimization

Xinyuan Wang, Chenxi Li, Zhen Wang et al.

202

ICLR 2024arXiv:2310.06117

#14

Take a Step Back: Evoking Reasoning via Abstraction in Large Language Models

Huaixiu Steven Zheng, Swaroop Mishra, Xinyun Chen et al.

190

ICML 2025arXiv:2404.16873

#15

AdvPrompter: Fast Adaptive Adversarial Prompting for LLMs

Anselm Paulus, Arman Zharmagambetov, Chuan Guo et al.

adversarial promptingjailbreaking attackslarge language modelsprompt optimization+4

132

ICLR 2024arXiv:2310.13345

#16

An LLM can Fool Itself: A Prompt-Based Adversarial Attack

Xilie Xu, Keyi Kong, Ning Liu et al.

131

ICML 2024arXiv:2312.04511

#17

An LLM Compiler for Parallel Function Calling

Sehoon Kim, Suhong Moon, Ryan Tabrizi et al.

parallel function callingllm reasoning capabilitiesfunction calling plannertask fetching unit+3

124

ICML 2024arXiv:2401.18018

#18

On Prompt-Driven Safeguarding for Large Language Models

Chujie Zheng, Fan Yin, Hao Zhou et al.

safety promptslarge language modelsrepresentation space analysisrefusal direction+3

106

ICLR 2025arXiv:2403.20271

#19

Draw-and-Understand: Leveraging Visual Prompts to Enable MLLMs to Comprehend What You Want

Weifeng Lin, Xinyu Wei, Ruichuan An et al.

visual promptingmultimodal large language modelspixel-level understandingmulti-domain dataset+4

87

ICLR 2025arXiv:2409.05907

#20

Programming Refusal with Conditional Activation Steering

Bruce W. Lee, Inkit Padhi, Karthikeyan Natesan Ramamurthy et al.

85

ICLR 2024arXiv:2402.19464

#21

Curiosity-driven Red-teaming for Large Language Models

Zhang-Wei Hong, Idan Shenfeld, Johnson (Tsun-Hsuan) Wang et al.

79

ICLR 2025arXiv:2310.11589

#22

Eliciting Human Preferences with Language Models

Belinda Li, Alex Tamkin, Noah Goodman et al.

language model promptinghuman preference alignmenttask specification elicitationinteractive learning frameworks+4

79

ICML 2024arXiv:2402.05863

#23

How Well Can LLMs Negotiate? NegotiationArena Platform and Analysis

Federico Bianchi, Patrick John Chia, Mert Yuksekgonul et al.

llm negotiation agentsmulti-agent dialogue systemstheory of mindbehavioral tactics+4

76

ICML 2024arXiv:2405.09783

#24

LLM and Simulation as Bilevel Optimizers: A New Paradigm to Advance Physical Scientific Discovery

Pingchuan Ma, Johnson Tsun-Hsuan Wang, Minghao Guo et al.

bilevel optimizationconstitutive law discoverymolecular designscientific generative agent+4

67

ICML 2024arXiv:2311.17371

#25

Should we be going MAD? A Look at Multi-Agent Debate Strategies for LLMs

Andries Smit, Nathan Grinsztajn, Paul Duckworth et al.

multi-agent debatelarge language modelsprompting strategiestruthfulness enhancement+2

64

ICLR 2024arXiv:2312.03724

#26

DP-OPT: Make Large Language Model Your Privacy-Preserving Prompt Engineer

Junyuan Hong, Jiachen (Tianhao) Wang, Chenhui Zhang et al.

62

ICML 2025arXiv:2501.18099

#27

Learning to Plan & Reason for Evaluation with Thinking-LLM-as-a-Judge

Swarnadeep Saha, Xian Li, Marjan Ghazvininejad et al.

60

ICLR 2025arXiv:2501.11120

#28

Tell me about yourself: LLMs are aware of their learned behaviors

Jan Betley, Xuchan Bao, Martín Soto et al.

behavioral self-awarenessai safetybackdoor policiesllm finetuning+2

59

ICML 2024arXiv:2402.05140

#29

Tag-LLM: Repurposing General-Purpose LLMs for Specialized Domains

Junhong Shen, Neil Tenenholtz, James Hall et al.

input tag conditioningdomain adaptationzero-shot generalizationspecialized domain tasks+4

54

ICLR 2025arXiv:2502.09597

#30

Do LLMs Recognize Your Preferences? Evaluating Personalized Preference Following in LLMs

Siyan Zhao, Mingyi Hong, Yang Liu et al.

51

ICLR 2025arXiv:2409.11136

#31

Promptriever: Instruction-Trained Retrievers Can Be Prompted Like Language Models

Orion Weller, Ben Van Durme, Dawn Lawrie et al.

instruction-tuned language modelsretrieval modelsinformation retrievalinstruction training+4

47

ICLR 2025arXiv:2407.00023

#32

Preble: Efficient Distributed Prompt Scheduling for LLM Serving

Vikranth Srivatsa, Zijian He, Reyna Abhyankar et al.

distributed llm servingkv state cachingprompt sharing optimizationcomputation load-balancing+3

46

AAAI 2024arXiv:2302.14691

#33

Investigating the Effectiveness of Task-Agnostic Prefix Prompt for Instruction Following

Seonghyeon Ye, Hyeonbin Hwang, Sohee Yang et al.

46

ICLR 2025arXiv:2406.06647

#34

How efficient is LLM-generated code? A rigorous & high-standard benchmark

Ruizhong Qiu, Weiliang Zeng, James Ezick et al.

program synthesiscode efficiency evaluationefficiency metric designllm-generated code+4

45

ICLR 2024arXiv:2309.06553

#35

Query-Dependent Prompt Evaluation and Optimization with Offline Inverse RL

Hao Sun, Alihan Hüyük, Mihaela van der Schaar

45

COLM 2025arXiv:2411.05025

#36

LLMs as Research Tools: A Large Scale Survey of Researchers’ Usage and Perceptions

Zhehui Liao, Maria Antoniak, Inyoung Cheong et al.

surveylarge language modelresearchsocietal impact

45

ICML 2025arXiv:2410.10630

#37

Thinking LLMs: General Instruction Following with Thought Generation

Tianhao Wu, Janice Lan, Weizhe Yuan et al.

44

AAAI 2024arXiv:2308.12033

#38

PREFER: Prompt Ensemble Learning via Feedback-Reflect-Refine

Chenrui Zhang, Lin Liu, Chuyuan Wang et al.

prompt ensemble learninglarge language modelsfeedback mechanismboosting algorithms+4

43

ICML 2025arXiv:2502.00640

#39

CollabLLM: From Passive Responders to Active Collaborators

Shirley Wu, Michel Galley, Baolin Peng et al.

large language modelsmultiturn collaborationreinforcement fine-tuninghuman-llm interaction+4

43

ICLR 2025arXiv:2411.02306

#40

On Targeted Manipulation and Deception when Optimizing LLMs for User Feedback

Marcus Williams, Micah Carroll, Adhyyan Narang et al.

reinforcement learninguser feedback optimizationtargeted manipulationdeceptive tactics+4

43

CVPR 2024arXiv:2309.05950

#41

Language Models as Black-Box Optimizers for Vision-Language Models

Shihong Liu, Samuel Yu, Zhiqiu Lin et al.

42

AAAI 2025arXiv:2401.02418

#42

Learning to Prompt with Text Only Supervision for Vision-Language Models

Muhammad Uzair Khattak, Muhammad Ferjad Naeem, Muzammal Naseer et al.

vision-language modelsprompt learningtext only supervisionlarge language models+4

42

ICLR 2024arXiv:2310.03965

#43

THOUGHT PROPAGATION: AN ANALOGICAL APPROACH TO COMPLEX REASONING WITH LARGE LANGUAGE MODELS

Junchi Yu, Ran He, Rex Ying

41

ICLR 2025arXiv:2410.12361

#44

Proactive Agent: Shifting LLM Agents from Reactive Responses to Active Assistance

Yaxi Lu, Shenzhi Yang, Cheng Qian et al.

41

ICML 2024arXiv:2402.05403

#45

In-Context Principle Learning from Mistakes

Tianjun Zhang, Aman Madaan, Luyu Gao et al.

in-context learningfew-shot promptingmulti-hop question answeringtextual question answering+4

40

AAAI 2025arXiv:2408.08781

#46

Evaluating the Evaluator: Measuring LLMs’ Adherence to Task Evaluation Instructions

Bhuvanashree Murugadoss, Christian Poelitz, Ian Drosos et al.

llms-as-a-judgeautomatic evaluationhuman preference alignmentevaluation instruction adherence+4

39

CVPR 2024arXiv:2312.04302

#47

Prompt Highlighter: Interactive Control for Multi-Modal LLMs

Yuechen Zhang, Shengju Qian, Bohao Peng et al.

38

ICLR 2025arXiv:2410.13788

#48

Modeling Future Conversation Turns to Teach LLMs to Ask Clarifying Questions

Michael Zhang, W. Bradley Knox, Eunsol Choi

clarifying question generationambiguous user requestspreference labeling methodsfuture conversation modeling+4

37

AAAI 2025arXiv:2403.02738

#49

Causal Prompting: Debiasing Large Language Model Prompting Based on Front-Door Adjustment

Congzhi Zhang, Linhai Zhang, Jialong Wu et al.

36

COLM 2025arXiv:2504.14225

#50

Know Me, Respond to Me: Benchmarking LLMs for Dynamic User Profiling and Personalized Responses at Scale

Bowen Jiang, Zhuoqun Hao, Young Min Cho et al.

personalizationlong contextmemoryconversational chatbot

36

ICLR 2025arXiv:2412.13337

#51

Unveiling the Secret Recipe: A Guide For Supervised Fine-Tuning Small LLMs

Aldo Pareja, Nikhil Shivakumar Nayak, Hao Wang et al.

supervised fine-tuninginstruction-tuning datasetssmall language modelstraining configurations+4

34

ICLR 2025arXiv:2410.08105

#52

What Makes Large Language Models Reason in (Multi-Turn) Code Generation?

Kunhao Zheng, Juliette Decugis, Jonas Gehring et al.

large language modelscode generationprompting techniqueschain-of-thought+3

34

AAAI 2025arXiv:2407.00737

#53

LLM4GEN: Leveraging Semantic Representation of LLMs for Text-to-Image Generation

Mushui Liu, Yuhang Ma, Zhen Yang et al.

text-to-image generationdiffusion modelssemantic representationlarge language models+4

33

ICLR 2025arXiv:2410.06961

#54

Self-Boosting Large Language Models with Synthetic Preference Data

Qingxiu Dong, Li Dong, Xingxing Zhang et al.

synthetic preference datamodel alignmentinstruction-following abilitiesself-boosting paradigm+4

32

AAAI 2025arXiv:2402.17564

#55

Unleashing the Potential of Large Language Models as Prompt Optimizers: Analogical Analysis with Gradient-based Model Optimizers

Xinyu Tang, Xiaolei Wang, Wayne Xin Zhao et al.

30

ECCV 2024arXiv:2403.11755

#56

Meta-Prompting for Automating Zero-shot Visual Recognition with LLMs

Muhammad Jehanzeb Mirza, Leonid Karlinsky, Wei Lin et al.

30

ICLR 2025arXiv:2409.20296

#57

PersonalLLM: Tailoring LLMs to Individual Preferences

Thomas Zollo, Andrew Siah, Naimeng Ye et al.

personalized language modelspreference alignmentreward model simulationin-context learning+3

28

ICML 2024arXiv:2402.07282

#58

How do Large Language Models Navigate Conflicts between Honesty and Helpfulness?

Ryan Liu, Theodore R Sumers, Ishita Dasgupta et al.

large language modelshuman preference optimizationchain-of-thought promptinghonesty-helpfulness trade-off+3

28

ICLR 2025arXiv:2410.12952

#59

Facilitating Multi-turn Function Calling for LLMs via Compositional Instruction Tuning

Mingyang Chen, sunhaoze, Tianpeng Li et al.

function callinginstruction tuningmulti-turn interactionscompositional tasks+3

26

ICLR 2025arXiv:2405.14804

#60

Can LLMs Solve Longer Math Word Problems Better?

Xin Xu, Tong Xiao, Zitong Chao et al.

math word problemscontext length generalizabilitylarge language modelsinstructional prompting+4

26

NEURIPS 2025arXiv:2503.22674

#61

QuestBench: Can LLMs ask the right question to acquire information in reasoning tasks?

Belinda Li, Been Kim, Zi Wang

24

ICML 2024arXiv:2308.07120

#62

Position: Key Claims in LLM Research Have a Long Tail of Footnotes

Anna Rogers, Sasha Luccioni

large language modelsemergent propertiesmodel definitioncritical examination+2

24

AAAI 2024arXiv:2312.07398

#63

LLMEval: A Preliminary Study on How to Evaluate Large Language Models

Yue Zhang, Ming Zhang, HaiPeng Yuan et al.

23

ICLR 2024arXiv:2402.11140

#64

Boosting of Thoughts: Trial-and-Error Problem Solving with Large Language Models

Sijia Chen, Baochun Li, Di Niu

22

ICML 2024arXiv:2402.06625

#65

Understanding the Effects of Iterative Prompting on Truthfulness

Satyapriya Krishna, Chirag Agarwal, Himabindu Lakkaraju

iterative promptingllm truthfulnessresponse calibrationprompting variants+2

21

AAAI 2024arXiv:2401.17390

#66

Customizing Language Model Responses with Contrastive In-Context Learning

Xiang Gao, Kamalika Das

contrastive learninglanguage model alignmentin-context learningintent customization+4

20

ECCV 2024arXiv:2407.08966

#67

LAPT: Label-driven Automated Prompt Tuning for OOD Detection with Vision-Language Models

Yabin Zhang, Wenjie Zhu, Chenhang He et al.

19

NEURIPS 2025arXiv:2405.16276

#68

Mechanism Design for LLM Fine-tuning with Multiple Reward Models

Haoran Sun, Yurong Chen, Siwei Wang et al.

19

ICLR 2025arXiv:2412.19394

#69

Generating Novel Leads for Drug Discovery Using LLMs with Logical Feedback

Shreyas Bhat Brahmavar, Ashwin Srinivasan, Tirtharaj Dash et al.

An Engorgio Prompt Makes Large Language Model Babble on

Jianshuo Dong, Ziyuan Zhang, Qingjie Zhang et al.

17

COLM 2025arXiv:2503.12854

#71

Enhancing LLM Reasoning with Iterative DPO: A Comprehensive Empirical Investigation

Songjun Tu, Jiahao Lin, Xiangyu Tian et al.

LLM ReasoningIterative Optimization

17

ICML 2024arXiv:2407.00256

#72

One Prompt is not Enough: Automated Construction of a Mixture-of-Expert Prompts

Ruochen Wang, Sohyun An, Minhao Cheng et al.

instruction design automationmixture-of-expert paradigmin-context learningkernel regression theory+3

16

NEURIPS 2025arXiv:2505.11413

#73

CARES: Comprehensive Evaluation of Safety and Adversarial Robustness in Medical LLMs

Sijia Chen, Xiaomin Li, mengxue zhang et al.

16

ICLR 2025arXiv:2501.13545

#74

LLMs Can Plan Only If We Tell Them

Bilgehan Sel, Ruoxi Jia, Ming Jin

large language modelsautonomous planninglong-horizon planningalgorithm-of-thoughts+2

16

ICLR 2024arXiv:2309.00384

#75

BatchPrompt: Accomplish more with less

Jianzhe Lin, Maurice Diesendruck, Liang Du et al.

15

ICLR 2025arXiv:2410.03864

#76

DOTS: Learning to Reason Dynamically in LLMs via Optimal Reasoning Trajectories Search

Murong Yue, Wenlin Yao, Haitao Mi et al.

15

COLM 2025arXiv:2502.14860

#77

ALFA: Aligning LLMs to Ask Good Questions A Case Study in Clinical Reasoning

Shuyue Stella Li, Jimin Mun, Faeze Brahman et al.

Information SeekingQuestion AskingReliable LLMClinical Reasoning+1

15

ICML 2024arXiv:2404.06910

#78

Learning to Learn Better Visual Prompts

Fengxiang Wang, Wanrong Huang, Shaowu Yang et al.

Superposition Prompting: Improving and Accelerating Retrieval-Augmented Generation

Thomas Merth, Qichen Fu, Mohammad Rastegari et al.

retrieval-augmented generationlong context processingattention mechanism efficiencytransformer-based llms+3

ICML 2025arXiv:2410.05952

#80

Active Evaluation Acquisition for Efficient LLM Benchmarking

Yang Li, Jie Ma, Miguel Ballesteros et al.

CVPR 2025arXiv:2502.19844

#81

ProAPO: Progressively Automatic Prompt Optimization for Visual Classification

Xiangyan Qu, Gaopeng Gou, Jiamin Zhuang et al.

ICLR 2024arXiv:2310.13855

#82

Evoke: Evoking Critical Thinking Abilities in LLMs via Reviewer-Author Prompt Editing

Xinyu Hu, Pengfei Tang, Simiao Zuo et al.

ICLR 2025arXiv:2410.07627

#83

Automatic Curriculum Expert Iteration for Reliable LLM Reasoning

Zirui Zhao, Hanze Dong, Amrita Saha et al.

COLM 2025arXiv:2504.01282

#84

Prompt-Reverse Inconsistency: LLM Self-Inconsistency Beyond Generative Randomness and Prompt Paraphrasing

Jihyun Janice Ahn, Wenpeng Yin

Large Language ModelNatural Language ProcessInconsistency of LLMsPrompt-Reverse Inconsistency+2

ICLR 2025arXiv:2410.03168

#85

Can Watermarked LLMs be Identified by Users via Crafted Prompts?

Aiwei Liu, Sheng Guan, Yiming Liu et al.

text watermarkinglarge language modelswatermark imperceptibilitywatermark identification+3

12

NEURIPS 2025arXiv:2411.14133

#86

GASP: Efficient Black-Box Generation of Adversarial Suffixes for Jailbreaking LLMs

Advik Basani, Xiao Zhang

jailbreak attacksadversarial suffixeslatent bayesian optimizationblack-box generation+4

12

NEURIPS 2025arXiv:2503.16024

#87

The Lighthouse of Language: Enhancing LLM Agents via Critique-Guided Improvement

Ruihan Yang, Fanghua Ye, Jian Li et al.

11

AAAI 2025arXiv:2403.05636

#88

Tuning-Free Accountable Intervention for LLM Deployment – a Metacognitive Approach

Zhen Tan, Jie Peng, Song Wang et al.

11

ICCV 2025arXiv:2412.15156

#89

Prompt-A-Video: Prompt Your Video Diffusion Model via Preference-Aligned LLM

Yatai Ji, Jiacheng Zhang, Jie Wu et al.

text-to-video modelsprompt adaptationvideo diffusion modeldirect preference optimization+4

10

ICML 2025arXiv:2408.09121

#90

Selective Prompt Anchoring for Code Generation

Yuan Tian, Tianyi Zhang

code generationlarge language modelsattention dilutionprompt anchoring+2

10

COLM 2025arXiv:2504.10430

#91

LLM Can be a Dangerous Persuader: Empirical Study of Persuasion Safety in Large Language Models

Minqian Liu, Zhiyang Xu, Xinyi Zhang et al.

persuasionlarge language modelssafetyevaluation

10

ICLR 2025arXiv:2412.09722

#92

GReaTer: Gradients Over Reasoning Makes Smaller Language Models Strong Prompt Optimizers

Sarkar Snigdha Sarathi Das, Ryo Kamoi, Bo Pang et al.

9

ICLR 2025arXiv:2501.14371

#93

DRESSing Up LLM: Efficient Stylized Question-Answering via Style Subspace Editing

Xinyu Ma, Yifeng Xu, Yang Lin et al.

9

AAAI 2025arXiv:2403.16427

#94

Re2LLM: Reflective Reinforcement Large Language Model for Session-based Recommendation

Ziyan Wang, Yingpeng Du, Zhu Sun et al.

session-based recommendationlarge language modelsreflective explorationknowledge base construction+3

AAAI 2025arXiv:2501.01679

#95

Adaptive Few-shot Prompting for Machine Translation with Pre-trained Language Models

Lei Tang, Jinghui Qin, Wenxuan Ye et al.

in-context learningmachine translationfew-shot promptingdemonstration retrieval+4

ICLR 2025arXiv:2407.12878

#96

Do LLMs have Consistent Values?

Naama Rozen, Liat Bezalel, Gal Elidan et al.

NEURIPS 2025arXiv:2502.20432

#97

LLM Strategic Reasoning: Agentic Study through Behavioral Game Theory

Jingru Jia, Zehua Yuan, Junhao Pan et al.

strategic decision-makingbehavioral game theorylarge language modelsnash equilibrium approximation+4

ICLR 2025arXiv:2410.05434

#98

Better than Your Teacher: LLM Agents that learn from Privileged AI Feedback

Sanjiban Choudhury, Paloma Sodhi

CVPR 2025arXiv:2411.01492

#99

EEE-Bench: A Comprehensive Multimodal Electrical And Electronics Engineering Benchmark

Ming Li, Jike Zhong, Tianle Chen et al.

multimodal benchmarklarge multimodal modelsengineering problem solvingvisual-textual integration+4

7

ICLR 2025arXiv:2502.04485

#100

Active Task Disambiguation with LLMs

Katarzyna Kobalczyk, Nicolás Astorga, Tennison Liu et al.

task ambiguitybayesian experimental designclarifying questionsmeta-cognitive reasoning+3

7