Exploration in RL

ICLR 2024arXiv:2310.00166

#2

Motif: Intrinsic Motivation from Artificial Intelligence Feedback

Martin Klissarov, Pierluca D'Oro, Shagun Sodhani et al.

91

ICLR 2025arXiv:2405.21046

#3

Exploratory Preference Optimization: Harnessing Implicit Q*-Approximation for Sample-Efficient RLHF

Tengyang Xie, Dylan Foster, Akshay Krishnamurthy et al.

79

ICLR 2024arXiv:2310.08887

#4

METRA: Scalable Unsupervised RL with Metric-Aware Abstraction

Seohong Park, Oleh Rybkin, Sergey Levine

75

NEURIPS 2025arXiv:2504.07954

#5

Perception-R1: Pioneering Perception Policy with Reinforcement Learning

En Yu, Kangheng Lin, Liang Zhao et al.

62

CVPR 2024arXiv:2312.02976

#6

SPOC: Imitating Shortest Paths in Simulation Enables Effective Navigation and Manipulation in the Real World

Kiana Ehsani, Tanmay Gupta, Rose Hendrix et al.

54

ICLR 2024arXiv:2310.19668

#7

DrM: Mastering Visual Reinforcement Learning through Dormant Ratio Minimization

Guowei Xu, Ruijie Zheng, Yongyuan Liang et al.

46

reinforcement learning scalingtool-integrated reasoningspontaneous code executionmathematical problem solving+4

#8

Agentic RL Scaling Law: Spontaneous Code Execution for Mathematical Problem Solving

Xinji Mai, Haotian Xu, Xing W et al.

NEURIPS 2025

40

ICML 2024arXiv:2402.00396

#9

Efficient Exploration for LLMs

Vikranth Dwaracherla, Seyed Mohammad Asghari, Botao Hao et al.

efficient explorationlarge language modelshuman feedbackreward modeling+3

37

ICLR 2025arXiv:2410.02052

#10

ExACT: Teaching AI Agents to Explore with Reflective-MCTS and Exploratory Learning

Xiao Yu, Baolin Peng, Vineeth Vajipey et al.

vision-language modelsautonomous agentsmonte carlo tree searchtest-time search+4

37

ICML 2024arXiv:2401.09750

#11

Exploration and Anti-Exploration with Distributional Random Network Distillation

Kai Yang, jian tao, Jiafei Lyu et al.

exploration in reinforcement learningrandom network distillationdistributional reinforcement learningbonus allocation+3

34

ICLR 2024arXiv:2305.18246

#12

Provable and Practical: Efficient Exploration in Reinforcement Learning via Langevin Monte Carlo

Haque Ishfaq, Qingfeng Lan, Pan Xu et al.

33

ICLR 2025arXiv:2405.15143

#13

Intelligent Go-Explore: Standing on the Shoulders of Giant Foundation Models

Cong Lu, Shengran Hu, Jeff Clune

hard-exploration problemsfoundation modelsrobotic controlreinforcement learning+4

27

ICML 2025arXiv:2410.06238

#14

EVOLvE: Evaluating and Optimizing LLMs For In-Context Exploration

Allen Nie, Yi Su, Bo Chang et al.

27

NEURIPS 2025arXiv:2505.16673

#15

R1-ShareVL: Incentivizing Reasoning Capabilities of Multimodal Large Language Models via Share-GRPO

Huanjin Yao, Qixiang Yin, Jingyi Zhang et al.

26

ICLR 2025arXiv:2412.12098

#16

MaxInfoRL: Boosting exploration in reinforcement learning through information gain maximization

Bhavya, Stelian Coros, Andreas Krause et al.

24

ICLR 2025arXiv:2412.17256

#17

B-STaR: Monitoring and Balancing Exploration and Exploitation in Self-Taught Reasoners

Weihao Zeng, Yuzhen Huang, Lulu Zhao et al.

23

ICLR 2025arXiv:2501.12735

#18

Online Preference Alignment for Language Models via Count-based Exploration

Chenjia Bai, Yang Zhang, Shuang Qiu et al.

reinforcement learning from human feedbackonline preference alignmentlarge language modelscount-based exploration+3

20

ICML 2024arXiv:2306.16978

#19

Learning Coverage Paths in Unknown Environments with Deep Reinforcement Learning

Arvi Jonnarth, Jie Zhao, Michael Felsberg

coverage path planningunknown environment navigationdeep reinforcement learningegocentric map representation+4

18

ICML 2024arXiv:2402.10342

#20

Exploration-Driven Policy Optimization in RLHF: Theoretical Insights on Efficient Data Utilization

Yihan Du, Anna Winnicki, Gal Dalal et al.

reinforcement learning from human feedbackpolicy optimizationtrajectory-based comparison feedbackelliptical potential analysis+3

17

AAAI 2024arXiv:2308.11272

#21

FoX: Formation-Aware Exploration in Multi-Agent Reinforcement Learning

Yonghyeon Jo, Sunwoo Lee, Junghyuk Yum et al.

multi-agent reinforcement learningpartial observabilityexploration space scalabilityformation-based equivalence+4

16

ICLR 2025arXiv:2410.09486

#22

ActSafe: Active Exploration with Safety Constraints for Reinforcement Learning

Yarden As, Bhavya, Lenart Treven et al.

reinforcement learningsafe explorationmodel-based rlprobabilistic dynamics model+3

15

NEURIPS 2025arXiv:2503.14858

#23

1000 Layer Networks for Self-Supervised RL: Scaling Depth Can Enable New Goal-Reaching Capabilities

Kevin Wang, Ishaan Javali, Michał Bortkiewicz et al.

15

AAAI 2024arXiv:2312.11118

#24

Explaining Reinforcement Learning Agents through Counterfactual Action Outcomes

Yotam Amitai, Yael Friedler, Ofra Amir

14

NEURIPS 2025arXiv:2504.03206

#25

Enhancing Personalized Multi-Turn Dialogue with Curiosity Reward

Yanming Wan, Jiaxing Wu, Marwa Abdulhai et al.

personalized dialogue systemsmulti-turn reinforcement learningcuriosity reward mechanismuser modeling+4

NEURIPS 2025arXiv:2506.08745

#26

Consistent Paths Lead to Truth: Self-Rewarding Reinforcement Learning for LLM Reasoning

Kongcheng Zhang, QI YAO, Shunyu Liu et al.

self-rewarding reinforcement learninglarge language model reasoningreasoning trajectory consistencyintrinsic reward mechanism+4

AAAI 2024arXiv:2312.12145

#27

OVD-Explorer: Optimism Should Not Be the Sole Pursuit of Exploration in Noisy Environments

Jinyi Liu, Zhi Wang, Yan Zheng et al.

reinforcement learningoptimistic explorationcontinuous controlenvironmental stochasticity+3

ICLR 2024arXiv:2403.06826

#28

In-context Exploration-Exploitation for Reinforcement Learning

Zhenwen Dai, Federico Tomasi, Sina Ghiassian

ICLR 2025arXiv:2408.10635

#29

Strategist: Self-improvement of LLM Decision Making via Bi-Level Tree Search

Jonathan Light, Min Cai, Weiqin Chen et al.

ICLR 2025arXiv:2410.03618

#30

Open-World Reinforcement Learning over Long Short-Term Imagination

Jiajian Li, Qi Wang, Yunbo Wang et al.

ICLR 2025arXiv:2408.05804

#31

A Single Goal is All You Need: Skills and Exploration Emerge from Contrastive RL without Rewards, Demonstrations, or Subgoals

Grace Liu, Michael Tang, Benjamin Eysenbach

contrastive reinforcement learningskill emergencedirected explorationreward-free learning+2

ICML 2024arXiv:2402.01361

#32

To the Max: Reinventing Reward in Reinforcement Learning

Grigorii Veviurko, Wendelin Boehmer, Mathijs de Weerdt

reinforcement learningreward function designmax-reward optimizationgoal-reaching environments+2

ICML 2024arXiv:2405.18110

#33

Individual Contributions as Intrinsic Exploration Scaffolds for Multi-agent Reinforcement Learning

Xinran Li, Zifan LIU, Shibo Chen et al.

multi-agent reinforcement learningintrinsic rewardcredit assignmentbayesian surprise+4

ICML 2024arXiv:2407.09905

#34

Global Reinforcement Learning : Beyond Linear and Convex Rewards via Submodular Semi-gradient Methods

Riccardo De Santi, Manish Prajapat, Andreas Krause

global reinforcement learningsubmodular optimizationcurvature-dependent approximationexperiment design+4

ICML 2024arXiv:2405.03064

#35

RICE: Breaking Through the Training Bottlenecks of Reinforcement Learning with Explanation

Zelei Cheng, Xian Wu, Jiahao Yu et al.

reinforcement learningsparse rewardsexplanation methodsinitial state distribution+3

COLM 2025arXiv:2312.00267

#36

Sample Efficient Preference Alignment in LLMs via Active Exploration

Viraj Mehta, Syrine Belakaria, Vikramjeet Das et al.

Sample EfficientDPORLHFAlignment+2

ICLR 2024arXiv:2311.15951

#37

Replay across Experiments: A Natural Extension of Off-Policy RL

Dhruva Tirumala, Thomas Lampe, Jose Enrique Chen et al.

ICLR 2024arXiv:2305.01521

#38

Unlocking the Power of Representations in Long-term Novelty-based Exploration

Alaa Saade, Steven Kapturowski, Daniele Calandriello et al.

9

ICML 2024arXiv:2306.01276

#39

Symmetric Replay Training: Enhancing Sample Efficiency in Deep Reinforcement Learning for Combinatorial Optimization

Hyeonah Kim, Minsu Kim, Sungsoo Ahn et al.

deep reinforcement learningcombinatorial optimizationsample efficiencymolecular optimization+3

9

ICML 2024arXiv:2403.06571

#40

Scalable Online Exploration via Coverability

Philip Amortila, Dylan Foster, Akshay Krishnamurthy

reinforcement learning explorationpolicy optimization objectivesl1-coverage objectivelow coverability mdps+4

9

ICLR 2025arXiv:2406.01309

#41

REvolve: Reward Evolution with Large Language Models using Human Feedback

RISHI HAZRA, Alkis Sygkounas, Andreas Persson et al.

reward function designreinforcement learninglarge language modelshuman feedback integration+3

ICLR 2025arXiv:2501.15418

#42

Episodic Novelty Through Temporal Distance

Yuhua Jiang, Qihan Liu, Yiqin Yang et al.

AAAI 2024arXiv:2402.02097

#43

Settling Decentralized Multi-Agent Coordinated Exploration by Novelty Sharing

Haobin Jiang, Ziluo Ding, Zongqing Lu

multi-agent reinforcement learningcoordinated explorationnovelty sharingdecentralized cooperation+4

AAAI 2024arXiv:2305.11476

#44

Learning Diverse Risk Preferences in Population-Based Self-Play

Yuhua Jiang, Qihan Liu, Xiaoteng Ma et al.

reinforcement learningself-play algorithmsrisk preferencespolicy diversity+4

ICML 2024arXiv:2407.13755

#45

Random Latent Exploration for Deep Reinforcement Learning

Srinath Mahankali, Zhang-Wei Hong, Ayush Sekhari et al.

reinforcement learningexploration techniquesrandomized value functionsexploration bonuses+3

AAAI 2024arXiv:2402.17978

#46

Imagine, Initialize, and Explore: An Effective Exploration Method in Multi-Agent Reinforcement Learning

Zeyang Liu, Lipeng Wan, Xinrui Yang et al.

multi-agent reinforcement learningexploration methodsintrinsic rewardsrole-based learning+4

AAAI 2024arXiv:2110.05286

#47

Learning from Ambiguous Demonstrations with Self-Explanation Guided Reinforcement Learning

Yantian Zha, Lin Guan, Subbarao Kambhampati

ICML 2025arXiv:2410.18076

#48

Leveraging Skills from Unlabeled Prior Data for Efficient Online Exploration

Max Wilcoxson, Qiyang Li, Kevin Frans et al.

unsupervised pretrainingreinforcement learningoffline trajectory dataexploration strategies+4

ICML 2024arXiv:2405.16030

#49

Constrained Ensemble Exploration for Unsupervised Skill Discovery

Chenjia Bai, Rushuai Yang, Qiaosheng Zhang et al.

unsupervised reinforcement learningskill discoveryensemble explorationstate entropy maximization+3

NEURIPS 2025arXiv:2505.19850

#50

DISCOVER: Automated Curricula for Sparse-Reward Reinforcement Learning

Leander Diaz-Bone, Marco Bagatella, Jonas Hübotter et al.

NEURIPS 2025arXiv:2505.20128

#51

Iterative Self-Incentivization Empowers Large Language Models as Agentic Searchers

Zhengliang Shi, Lingyong Yan, Dawei Yin et al.

agentic search frameworkinformation retrievalmulti-hop queriesself-incentivized learning+2

ICML 2024arXiv:2402.18866

#52

Dr. Strategy: Model-Based Generalist Agents with Strategic Dreaming

Hany Hamed, Subin Kim, Dongyeong Kim et al.

model-based reinforcement learningsample efficiencygeneralist agentsstrategic dreaming+4

AAAI 2025arXiv:2412.16395

#53

Autonomous Option Invention for Continual Hierarchical Reinforcement Learning and Planning

Rashmeet Kaur Nayyar, Siddharth Srivastava

NEURIPS 2025arXiv:2501.18009

#54

Large Language Models Think Too Fast To Explore Effectively

Lan Pan, Hanbo Xie, Robert Wilson

large language modelsexploration capacityopen-ended tasksuncertainty-driven strategies+4

NEURIPS 2025arXiv:2507.02834

#55

ExPO: Unlocking Hard Reasoning with Self-Explanation-Guided Reinforcement Learning

Ruiyang Zhou, Shuozhe Li, Amy Zhang et al.

ICLR 2024arXiv:2312.08369

#56

UTILITY: Utilizing Explainable Reinforcement Learning to Improve Reinforcement Learning

Shicheng Liu, Minghui Zhu

The Effective Horizon Explains Deep RL Performance in Stochastic Environments

Cassidy Laidlaw, Banghua Zhu, Stuart Russell et al.

NEURIPS 2025arXiv:2505.18098

#58

Planning without Search: Refining Frontier LLMs with Offline Goal-Conditioned RL

Joey Hong, Anca Dragan, Sergey Levine

ICML 2025arXiv:2505.13652

#59

Guided Search Strategies in Non-Serializable Environments with Applications to Software Engineering Agents

Karina Zainullina, Aleksandr Golubev, Maria Trofimova et al.

ICML 2024arXiv:2406.16255

#60

Uncertainty-Aware Reward-Free Exploration with General Function Approximation

Junkai Zhang, Weitong Zhang, Dongruo Zhou et al.

reward-free reinforcement learninggeneral function approximationuncertainty-aware explorationintrinsic reward design+4

ICML 2025arXiv:2411.18825

#61

ELEMENTAL: Interactive Learning from Demonstrations and Vision-Language Models for Reward Design in Robotics

Letian Chen, Nina Moorman, Matthew Gombolay

ICLR 2024arXiv:2310.18144

#62

Improving Intrinsic Exploration by Creating Stationary Objectives

Roger Creus Castanyer, Joshua Romoff, Glen Berseth

ICML 2025arXiv:2503.01584

#63

SENSEI: Semantic Exploration Guided by Foundation Models to Learn Versatile World Models

Cansu Sancaktar, Christian Gumbsch, Andrii Zadaianchuk et al.

ICML 2025arXiv:2505.08735

#64

Preference Optimization for Combinatorial Optimization Problems

Mingjun Pan, Guanquan Lin, You-Wei Luo et al.

ICML 2025arXiv:2502.07709

#65

MAGELLAN: Metacognitive predictions of learning progress guide autotelic LLM agents in large goal spaces

Loris Gaven, Thomas Carta, Clément Romac et al.

ICML 2024arXiv:2308.13049

#66

Bayesian Exploration Networks

Mattie Fellows, Brandon Kaplowitz, Christian Schroeder de Witt et al.

bayesian reinforcement learningsequential decision makingexploration exploitation dilemmamodel-free approaches+4

ICML 2025arXiv:2409.15963

#67

Provably Efficient Exploration in Inverse Constrained Reinforcement Learning

Bo Yue, Jian Li, Guiliang Liu

ICLR 2025arXiv:2410.15184

#68

Action abstractions for amortized sampling

Oussama Boussif, Léna Ezzine, Joseph Viviano et al.

reinforcement learninggenerative flow networksaction abstractionshierarchical planning+4

NEURIPS 2025arXiv:2511.20066

#69

SOMBRL: Scalable and Optimistic Model-Based RL

Bhavya, Lenart Treven, Carmelo Sferrazza et al.

model-based reinforcement learningefficient explorationuncertainty-aware dynamicsoptimism in uncertainty+4

NEURIPS 2025arXiv:2505.15293

#70

LLM-Explorer: A Plug-in Reinforcement Learning Policy Exploration Enhancement Driven by Large Language Models

Qianyue Hao, Yiwen Song, Qingmin Liao et al.

reinforcement learningpolicy explorationlarge language modelsstochastic processes+4

ICML 2025arXiv:2412.16318

#71

Principal-Agent Bandit Games with Self-Interested and Exploratory Learning Agents

Junyan Liu, Lillian Ratliff

NEURIPS 2025arXiv:2506.06981

#72

Deep RL Needs Deep Behavior Analysis: Exploring Implicit Planning by Model-Free Agents in Open-Ended Environments

Riley Simmons-Edler, Ryan Badman, Felix Berg et al.

reinforcement learning agentsbehavioral analysismodel-free agentsimplicit planning+4

NEURIPS 2025arXiv:2506.09508

#73

Efficient Preference-Based Reinforcement Learning: Randomized Exploration meets Experimental Design

Andreas Schlaginhaufen, Reda Ouhamma, Maryam Kamgarpour

preference-based reinforcement learninghuman feedback learningmarkov decision processesrandomized exploration+4

reinforcement learning from human feedbackonline rlhfsample efficiencypreference-based exploration+3

#74

Avoiding exp(R) scaling in RLHF through Preference-based Exploration

Mingyu Chen, Yiding Chen, Wen Sun et al.

NEURIPS 2025

ICLR 2025arXiv:2410.04988

#75

Efficient Model-Based Reinforcement Learning Through Optimistic Thompson Sampling

Jasmine Bayrooti, Carl Ek, Amanda Prorok

model-based reinforcement learningoptimistic explorationthompson samplingcontinuous control tasks+4

ICML 2024arXiv:2407.13364

#76

Geometric Active Exploration in Markov Decision Processes: the Benefit of Abstraction

Riccardo De Santi, Federico Arangath Joseph, Noah Liniger et al.

active explorationmarkov decision processesmdp homomorphismsconvex reinforcement learning+4

NEURIPS 2025arXiv:2412.05718

#77

RLZero: Direct Policy Inference from Language Without In-Domain Supervision

Harshit Sushil Sikchi, Siddhant Agarwal, Pranaya Jajoo et al.

language-conditioned reinforcement learningzero-shot policy inferenceunsupervised rl pretrainingvideo generative models+3

ICCV 2025arXiv:2411.17764

#78

PROGRESSOR: A Perceptually Guided Reward Estimator with Self-Supervised Online Refinement

Tewodros W. Ayalew, Xiao Zhang, Kevin Y Wu et al.

reward function learninggoal-conditioned reinforcement learningself-supervised learningadversarial refinement+4

ICML 2025arXiv:2505.12204

#79

Of Mice and Machines: A Comparison of Learning Between Real World Mice and RL Agents

Shuo Han, German Espinosa, Junda Huang et al.

ICLR 2024arXiv:2403.01636

#80

Sample Efficient Myopic Exploration Through Multitask Reinforcement Learning with Diverse Tasks

Ziping Xu, Zifan Xu, Runxuan Jiang et al.

multitask reinforcement learningmyopic explorationsample efficiencydiverse tasks+4

AAAI 2024arXiv:2403.06880

#81

Unveiling the Significance of Toddler-Inspired Reward Transition in Goal-Oriented Reinforcement Learning

Junseok Park, Yoonsung Kim, Hee bin Yoo et al.

NEURIPS 2025arXiv:2506.04723

#82

Beyond Accuracy: Dissecting Mathematical Reasoning for LLMs Under Reinforcement Learning

Jiayu Wang, Yifei Ming, Zixuan Ke et al.

reinforcement learning tuningmathematical reasoningplan following executionknowledge integration+4

AAAI 2024arXiv:2307.05209

#83

Contextual Pre-planning on Reward Machine Abstractions for Enhanced Transfer in Deep Reinforcement Learning

Guy Azran, Mohamad H Danesh, Stefano Albrecht et al.

reward machinesdeep reinforcement learningtransfer learningsymbolic representations+3

multi-agent reinforcement learningdecentralized policiestrajectory entropy maximizationcontrastive trajectory representation+4

#84

Toward Efficient Multi-Agent Exploration With Trajectory Entropy Maximization

Tianxu Li, Kun Zhu

ICLR 2025

ICLR 2025arXiv:2506.20307

#85

Beyond-Expert Performance with Limited Demonstrations: Efficient Imitation Learning with Double Exploration

Heyang Zhao, Xingrui Yu, David Bossens et al.

AAAI 2025arXiv:2412.13492

#86

Efficient Language-instructed Skill Acquisition via Reward-Policy Co-Evolution

Changxin Huang, Yanbin Chang, Junfan Lin et al.

reward function designpolicy improvementlanguage-instructed skill acquisitionrobotic autonomy+4

NEURIPS 2025arXiv:2512.02423

#87

Abstract and Explore: A Novel Behavioral Metric with Cyclic Dynamics in Reinforcement Learning

Anjie Zhu, Peng-Fei Zhang, Ruihong Qiu et al.

GUI Exploration Lab: Enhancing Screen Navigation in Agents via Multi-Turn Reinforcement Learning

Haolong Yan, Yeqing Shen, Xin Huang et al.

gui agent navigationscreen navigation challengessimulation environment enginemulti-turn reinforcement learning+4

ICLR 2025arXiv:2408.11816

#89

Efficient Exploration and Discriminative World Model Learning with an Object-Centric Abstraction

Anthony GX-Chen, Kenneth Marino, Rob Fergus

ICLR 2025arXiv:2503.00810

#90

Minimax Optimal Reinforcement Learning with Quasi-Optimism

Harin Lee, Min-hwan Oh

NEURIPS 2025arXiv:2506.16396

#91

GoalLadder: Incremental Goal Discovery with Vision-Language Models

Alexey Zakharov, Shimon Whiteson

vision-language modelsreinforcement learningreward shapinggoal discovery+4

ICLR 2025arXiv:2502.20341

#92

Safety Representations for Safer Policy Learning

Kaustubh Mani, Vincent Mai, Charlie Gauthier et al.

safety representationssafe policy learningsafe explorationreinforcement learning+2

ICML 2025arXiv:2503.18234

#93

KEA: Keeping Exploration Alive by Proactively Coordinating Exploration Strategies

Shih-Min Yang, Martin Magnusson, Johannes Stork et al.

NEURIPS 2025arXiv:2506.00138

#94

Intrinsic Goals for Autonomous Agents: Model-Based Exploration in Virtual Zebrafish Predicts Ethological Behavior and Whole-Brain Dynamics

Reece Keller, Alyn Kirsch, Felix Pei et al.

model-based intrinsic motivationautonomous explorationwhole-brain dynamicsneural-glial computation+4

ICML 2025arXiv:2507.09041

#95

Behavioral Exploration: Learning to Explore via In-Context Adaptation

Andrew Wagenmaker, Zhiyuan Zhou, Sergey Levine

ICML 2025arXiv:2505.12611

#96

Action-Dependent Optimality-Preserving Reward Shaping

Grant Forbes, Jianxun Wang, Leonardo Villalobos-Arias et al.