Deep Reinforcement Learning

ICLR 2025arXiv:2307.15217

#2

Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback

Javier Rando, Tony Wang, Stewart Slocum et al.

reinforcement learning from human feedbacklarge language modelshuman alignmentai system safety+3

750

COLM 2025arXiv:2503.20783

#3

Understanding R1-Zero-Like Training: A Critical Perspective

Zichen Liu, Changyu Chen, Wenjun Li et al.

R1-ZeroReinforcement LearningPost-Training

714

NEURIPS 2025arXiv:2504.13837

#4

Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?

Yang Yue, Zhiqi Chen, Rui Lu et al.

reinforcement learning with verifiable rewardsreasoning capacitylarge language modelsmathematics tasks+4

540

ICML 2024arXiv:2309.00267

#5

RLAIF vs. RLHF: Scaling Reinforcement Learning from Human Feedback with AI Feedback

Harrison Lee, Samrat Phatale, Hassan Mansoor et al.

reinforcement learning from human feedbackreinforcement learning from ai feedbackreward model traininglarge language model alignment+4

527

ICML 2025arXiv:2501.17161

#6

SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training

Tianzhe Chu, Yuexiang Zhai, Jihan Yang et al.

442

COLM 2025arXiv:2503.18892

#7

SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wild

Weihao Zeng, Yuzhen Huang, Qian Liu et al.

ReasoningLarge Language Model

390

NEURIPS 2025arXiv:2503.24290

#8

Open-Reasoner-Zero: An Open Source Approach to Scaling Up Reinforcement Learning on the Base Model

Jingcheng Hu, Yinmin Zhang, Qi Han et al.

reinforcement learningbase model trainingreasoning-oriented rlppo with gae+4

347

ICML 2024arXiv:2312.11456

#9

Iterative Preference Learning from Human Feedback: Bridging Theory and Practice for RLHF under KL-constraint

Wei Xiong, Hanze Dong, Chenlu Ye et al.

reinforcement learning from human feedbackreverse-kl regularizationcontextual bandit formulationdirect preference optimization+4

312

ICLR 2024arXiv:2310.06452

#10

Understanding the Effects of RLHF on LLM Generalisation and Diversity

Robert Kirk, Ishita Mediratta, Christoforos Nalmpantis et al.

287

NEURIPS 2025arXiv:2503.21776

#11

Video-R1: Reinforcing Video Reasoning in MLLMs

Kaituo Feng, Kaixiong Gong, Bohao Li et al.

rule-based reinforcement learningmultimodal large language modelsvideo reasoningtemporal modeling+3

257

ICML 2024arXiv:2312.00886

#12

Nash Learning from Human Feedback

REMI MUNOS, Michal Valko, Daniele Calandriello et al.

reinforcement learning from human feedbackreward model learningpreference modelingnash equilibrium+4

195

NEURIPS 2025arXiv:2504.20571

#13

Reinforcement Learning for Reasoning in Large Language Models with One Training Example

Yiping Wang, Qing Yang, Zhiyuan Zeng et al.

190

NEURIPS 2025arXiv:2504.08837

#14

VL-Rethinker: Incentivizing Self-Reflection of Vision-Language Models with Reinforcement Learning

Haozhe Wang, Chao Qu, Zuming Huang et al.

vision-language modelsreinforcement learningslow-thinking systemsmultimodal reasoning+4

183

NEURIPS 2025arXiv:2504.13958

#15

ToolRL: Reward is All Tool Learning Needs

Cheng Qian, Emre Can Acikgoz, Qi He et al.

178

NEURIPS 2025arXiv:2502.18449

#16

SWE-RL: Advancing LLM Reasoning via Reinforcement Learning on Open Software Evolution

Yuxiang Wei, Olivier Duchenne, Jade Copet et al.

156

ICML 2024arXiv:2310.10505

#17

ReMax: A Simple, Effective, and Efficient Reinforcement Learning Method for Aligning Large Language Models

Ziniu Li, Tian Xu, Yushun Zhang et al.

reinforcement learning from human feedbacklarge language model alignmentproximal policy optimizationreinforce algorithm+4

147

NEURIPS 2025arXiv:2505.03335

#18

Absolute Zero: Reinforced Self-play Reasoning with Zero Data

Andrew Zhao, Yiran Wu, Yang Yue et al.

147

ICLR 2024arXiv:2310.12921

#19

Vision-Language Models are Zero-Shot Reward Models for Reinforcement Learning

Juan Rocamonde, Victoriano Montesinos, Elvis Nava et al.

137

ICML 2024arXiv:2402.19446

#20

ArCHer: Training Language Model Agents via Hierarchical Multi-Turn RL

Yifei Zhou, Andrea Zanette, Jiayi Pan et al.

language model agentssequential decision-makingmulti-turn reinforcement learningtoken-level policy gradient+4

135

NEURIPS 2025arXiv:2504.16084

#21

TTRL: Test-Time Reinforcement Learning

Yuxin Zuo, Kaiyan Zhang, Li Sheng et al.

test-time reinforcement learningreward estimationlarge language modelsreasoning tasks+4

129

NEURIPS 2025arXiv:2504.14945

#22

Learning to Reason under Off-Policy Guidance

Jianhao Yan, Yafu Li, Zican Hu et al.

129

ICML 2024arXiv:2402.10207

#23

Rewards-in-Context: Multi-objective Alignment of Foundation Models with Dynamic Preference Adjustment

Rui Yang, Xiaoman Pan, Feng Luo et al.

multi-objective alignmenthuman preference alignmentreinforcement learning fine-tuningsupervised fine-tuning+4

125

ICLR 2025arXiv:2411.02337

#24

WebRL: Training LLM Web Agents via Self-Evolving Online Curriculum Reinforcement Learning

Zehan Qi, Xiao Liu, Iat Long Iong et al.

llm web agentsonline curriculum reinforcement learningself-evolving curriculumoutcome-supervised reward model+3

121

ICML 2024arXiv:2402.03681

#25

RL-VLM-F: Reinforcement Learning from Vision Language Foundation Model Feedback

Yufei Wang, Zhanyi Sun, Jesse Zhang et al.

reward engineeringvision language modelspreference learningautonomous manipulation+4

119

ICML 2024arXiv:2310.17022

#26

Controlled Decoding from Language Models

Sidharth Mudgal, Jong Lee, Harish Ganapathy et al.

language model alignmentkl-regularized reinforcement learningcontrolled decodingprefix scorer+3

118

NEURIPS 2025arXiv:2505.24298

#27

AREAL: A Large-Scale Asynchronous Reinforcement Learning System for Language Reasoning

Wei Fu, Jiaxuan Gao, Xujie Shen et al.

reinforcement learninglarge language modelsreasoning tasksasynchronous training+4

117

ICLR 2024arXiv:2309.11489

#28

Text2Reward: Reward Shaping with Language Models for Reinforcement Learning

Tianbao Xie, Siheng Zhao, Chen Henry Wu et al.

111

ICML 2024arXiv:2403.03950

#29

Stop Regressing: Training Value Functions via Classification for Scalable Deep RL

Jesse Farebrother, Jordi Orbay, Quan Vuong et al.

value functionsdeep reinforcement learningcategorical cross-entropyscalable rl+4

107

NEURIPS 2025arXiv:2505.24864

#30

ProRL: Prolonged Reinforcement Learning Expands Reasoning Boundaries in Large Language Models

Mingjie Liu, Shizhe Diao, Ximing Lu et al.

reinforcement learningreasoning capabilitieskl divergence controlreference policy resetting+4

104

ICLR 2024arXiv:2312.08358

#31

Distributional Preference Learning: Understanding and Accounting for Hidden Context in RLHF

Anand Siththaranjan, Cassidy Laidlaw, Dylan Hadfield-Menell

99

ICML 2025arXiv:2410.02089

#32

RLEF: Grounding Code LLMs in Execution Feedback with Reinforcement Learning

Jonas Gehring, Kunhao Zheng, Jade Copet et al.

91

NEURIPS 2025arXiv:2506.01347

#33

The Surprising Effectiveness of Negative Reinforcement in LLM Reasoning

Xinyu Zhu, Mengzhou Xia, Zhepei Wei et al.

reinforcement learningmathematical reasoninglanguage modelspolicy gradients+4

89

ICML 2024arXiv:2402.08925

#34

MaxMin-RLHF: Alignment with Diverse Human Preferences

Souradip Chakraborty, Jiahao Qiu, Hui Yuan et al.

reinforcement learning from human feedbackreward model diversityhuman preference alignmentmixture of reward models+3

88

NEURIPS 2025arXiv:2504.12216

#35

d1: Scaling Reasoning in Diffusion Large Language Models via Reinforcement Learning

Siyan Zhao, Devaansh Gupta, Qinqing Zheng et al.

diffusion large language modelsreinforcement learningnon-autoregressive generationreasoning capabilities+4

87

COLM 2025arXiv:2504.07912

#36

Echo Chamber: RL Post-training Amplifies Behaviors Learned in Pretraining

Rosie Zhao, Alexandru Meterez, Sham M. Kakade et al.

reinforcement learninglanguage modelspost-trainingppo+2

87

NEURIPS 2025arXiv:2505.14652

#37

General-Reasoner: Advancing LLM Reasoning Across All Domains

Xueguang Ma, Qian Liu, Dongfu Jiang et al.

reinforcement learninglarge language modelsreasoning capabilitiesanswer verification+4

86

ICLR 2024arXiv:2305.14483

#38

Language Model Self-improvement by Reinforcement Learning Contemplation

Jing-Cheng Pang, Pengyuan Wang, Kaiyuan Li et al.

81

ICLR 2024arXiv:2312.10812

#39

Learning to Act without Actions

Dominik Schmidt, Minqi Jiang

78

ECCV 2024arXiv:2401.12244

#40

Large-scale Reinforcement Learning for Diffusion Models

Yinan Zhang, Eric Tzeng, Yilun Du et al.

reinforcement learningdiffusion modelshuman preference alignmenttext-to-image generation+3

77

ICLR 2024arXiv:2405.01534

#41

Plan-Seq-Learn: Language Model Guided RL for Solving Long Horizon Robotics Tasks

Murtaza Dalal, Tarun Chiruvolu, Devendra Chaplot et al.

76

ICLR 2024arXiv:2310.08887

#42

METRA: Scalable Unsupervised RL with Metric-Aware Abstraction

Seohong Park, Oleh Rybkin, Sergey Levine

75

ICLR 2024arXiv:2310.08566

#43

Transformers as Decision Makers: Provable In-Context Reinforcement Learning via Supervised Pretraining

Licong Lin, Yu Bai, Song Mei

70

ICML 2025arXiv:2311.18232

#44

LMRL Gym: Benchmarks for Multi-Turn Reinforcement Learning with Language Models

Marwa Abdulhai, Isadora White, Charlie Snell et al.

67

ICML 2024arXiv:2402.00782

#45

Dense Reward for Free in Reinforcement Learning from Human Feedback

Alexander Chan, Hao Sun, Samuel Holt et al.

reinforcement learning from human feedbackreward model attentionpotential-based reward shapingsparse reward problem+3

65

ICML 2024arXiv:2402.08609

#46

Mixtures of Experts Unlock Parameter Scaling for Deep RL

Johan Obando Ceron, Ghada Sokar, Timon Willi et al.

mixture of expertsparameter scalingreinforcement learningvalue-based networks+2

64

NEURIPS 2025arXiv:2504.07954

#47

Perception-R1: Pioneering Perception Policy with Reinforcement Learning

En Yu, Kangheng Lin, Liang Zhao et al.

62

ICML 2024arXiv:2402.15567

#48

Foundation Policies with Hilbert Representations

Seohong Park, Tobias Kreiman, Sergey Levine

unsupervised pre-trainingreinforcement learning policiesoffline datazero-shot adaptation+4

59

CVPR 2024arXiv:2312.09238

#49

Auto MC-Reward: Automated Dense Reward Design with Large Language Models for Minecraft

Hao Li, Xue Yang, Zhaokai Wang et al.

58

ICML 2024arXiv:2402.05808

#50

Training Large Language Models for Reasoning through Reverse Curriculum Reinforcement Learning

Zhiheng Xi, Wenxiang Chen, Boyang Hong et al.

reinforcement learninglarge language modelsreasoning tasksoutcome supervision+4

58

NEURIPS 2025arXiv:2504.13055

#51

NoisyRollout: Reinforcing Visual Reasoning with Data Augmentation

Xiangyan Liu, Jinjie Ni, Zijian Wu et al.

57

NEURIPS 2025arXiv:2503.19470

#52

ReSearch: Learning to Reason with Search for LLMs via Reinforcement Learning

Mingyang Chen, Linzhuang Sun, Tianpeng Li et al.

reasoning with searchreinforcement learningmulti-hop question answeringsearch-guided reasoning+3

57

ICLR 2025arXiv:2407.04811

#53

Simplifying Deep Temporal Difference Learning

Matteo Gallici, Mattie Fellows, Benjamin Ellis et al.

temporal difference learningoff-policy learningq-learning algorithmsreinforcement learning stability+4

56

ICLR 2025arXiv:2410.14803

#54

DistRL: An Asynchronous Distributed Reinforcement Learning Framework for On-Device Control Agent

Taiyi Wang, Zhihao Wu, Jianheng Liu et al.

54

NEURIPS 2025arXiv:2505.16400

#55

Training Diffusion Models Towards Diverse Image Generation with Reinforcement Learning

Zichen Miao, Jiang Wang, Ze Wang et al.

AceReason-Nemotron: Advancing Math and Code Reasoning through Reinforcement Learning

Yang Chen, Zhuolin Yang, Zihan Liu et al.

50

ICLR 2024arXiv:2306.09884

#57

Jumanji: a Diverse Suite of Scalable Reinforcement Learning Environments in JAX

Clément Bonnet, Daniel Luo, Donal Byrne et al.

48

ICML 2024arXiv:2401.16335

#58

Iterative Data Smoothing: Mitigating Reward Overfitting and Overoptimization in RLHF

Banghua Zhu, Michael Jordan, Jiantao Jiao

reward model learninghuman feedback alignmentreward overoptimizationiterative data smoothing+3

48

ICLR 2025arXiv:2410.09754

#59

SimBa: Simplicity Bias for Scaling Up Parameters in Deep Reinforcement Learning

Hojoon Lee, Dongyoon Hwang, Donghu Kim et al.

47

ICLR 2024arXiv:2310.19668

#60

DrM: Mastering Visual Reinforcement Learning through Dormant Ratio Minimization

Guowei Xu, Ruijie Zheng, Yongyuan Liang et al.

46

NEURIPS 2025arXiv:2505.19789

#61

What Can RL Bring to VLA Generalization? An Empirical Study

Jijia Liu, Feng Gao, Bingwen Wei et al.

46

NEURIPS 2025arXiv:2502.13144

#62

RAD: Training an End-to-End Driving Policy via Large-Scale 3DGS-based Reinforcement Learning

Hao Gao, Shaoyu Chen, Bo Jiang et al.

45

ICLR 2024arXiv:2401.08898

#63

Bridging State and History Representations: Understanding Self-Predictive RL

Tianwei Ni, Benjamin Eysenbach, Erfan Seyedsalehi et al.

ECCV 2024arXiv:2312.14134

#64

Diffusion Reward: Learning Rewards via Conditional Video Diffusion

Tao Huang, Guangqi Jiang, Yanjie Ze et al.

reward learningexpert video conditioningconditional video diffusionvisual reinforcement learning+4

NEURIPS 2025arXiv:2507.07966

#65

Scaling RL to Long Videos

Yukang Chen, Wei Huang, Baifeng Shi et al.

vision-language modelslong video reasoningreinforcement learningchain-of-thought fine-tuning+4

Large Language ModelReinforcement LearningMathmatical Reasoning

#66

Exploring the Limit of Outcome Reward for Learning Mathematical Reasoning

Chengqi Lyu, Songyang Gao, Yuzhe Gu et al.

COLM 2025

ICLR 2024arXiv:2305.14816

#67

Provable Offline Preference-Based Reinforcement Learning

Wenhao Zhan, Masatoshi Uehara, Nathan Kallus et al.

43

reinforcement learningautonomous drivingworld modellatent state space+4

#68

Think2Drive: Efficient Reinforcement Learning by Thinking with Latent World Model for Autonomous Driving (in CARLA-v2)

Qifeng Li, Xiaosong Jia, Shaobo Wang et al.

ECCV 2024

43

ICML 2024arXiv:2403.00514

#69

Overestimation, Overfitting, and Plasticity in Actor-Critic: the Bitter Lesson of Reinforcement Learning

Michal Nauman, Michał Bortkiewicz, Piotr Milos et al.

off-policy reinforcement learningactor-critic methodsregularization techniquesoverestimation bias+3

41

ICLR 2024arXiv:2302.08560

#70

Dual RL: Unification and New Methods for Reinforcement and Imitation Learning

Harshit Sikchi, Qinqing Zheng, Amy Zhang et al.

41

NEURIPS 2025arXiv:2506.14965

#71

Revisiting Reinforcement Learning for LLM Reasoning from A Cross-Domain Perspective

Jorge (Zhoujun) Cheng, Shibo Hao, Tianyang Liu et al.

reinforcement learninglarge language model reasoningcross-domain trainingreward signal design+4

40

ICLR 2024arXiv:2403.04253

#72

Mastering Memory Tasks with World Models

Mohammad Reza Samsami, Artem Zholus, Janarthanan Rajendran et al.

40

reinforcement learning scalingtool-integrated reasoningspontaneous code executionmathematical problem solving+4

#73

Agentic RL Scaling Law: Spontaneous Code Execution for Mathematical Problem Solving

Xinji Mai, Haotian Xu, Xing W et al.

NEURIPS 2025

40

NEURIPS 2025arXiv:2505.10832

#74

Learning When to Think: Shaping Adaptive Reasoning in R1-Style Models via Multi-Stage RL

Songjun Tu, Jiahao Lin, Qichao Zhang et al.

39

COLM 2025arXiv:2503.00223

#75

DeepRetrieval: Hacking Real Search Engines and Retrievers with Large Language Models via Reinforcement Learning

Pengcheng Jiang, Jiacheng Lin, Lang Cao et al.

Large Language ModelsInformation RetrievalReinforcement Learning

38

ICLR 2024arXiv:2307.07176

#76

SafeDreamer: Safe Reinforcement Learning with World Models

Weidong Huang, Jiaming Ji, Chunhe Xia et al.

safe reinforcement learningworld model planninglagrangian-based methodsvision-only tasks+1

37

ICLR 2024arXiv:2309.06599

#77

Reasoning with Latent Diffusion in Offline Reinforcement Learning

Siddarth Venkatraman, Shivesh Khaitan, Ravi Tej Akella et al.

36

ICML 2024arXiv:2402.17257

#78

RIME: Robust Preference-based Reinforcement Learning with Noisy Preferences

Jie Cheng, Gang Xiong, Xingyuan Dai et al.

preference-based reinforcement learningnoisy preference learningrobust reward learningsample selection discriminator+3

35

ICCV 2025arXiv:2503.15265

#79

DeepMesh: Auto-Regressive Artist-mesh Creation with Reinforcement Learning

Ruowen Zhao, James Jun Liang Chen Ye, Zhengyi Wang et al.

triangle mesh generationauto-regressive modelingmesh tokenization algorithmreinforcement learning for 3d+4

35

ICML 2024arXiv:2401.09750

#80

Exploration and Anti-Exploration with Distributional Random Network Distillation

Kai Yang, jian tao, Jiafei Lyu et al.

exploration in reinforcement learningrandom network distillationdistributional reinforcement learningbonus allocation+3

ICLR 2025arXiv:2504.11054

#81

Zero-Shot Whole-Body Humanoid Control via Behavioral Foundation Models

Andrea Tirinzoni, Ahmed Touati, Jesse Farebrother et al.

NEURIPS 2025arXiv:2505.15810

#82

GUI-G1: Understanding R1-Zero-Like Training for Visual Grounding in GUI Agents

Yuqi Zhou, Sunhao Dai, Shuai Wang et al.

CVPR 2024arXiv:2402.08714

#83

PRDP: Proximal Reward Difference Prediction for Large-Scale Reward Finetuning of Diffusion Models

Fei Deng, Qifei Wang, Wei Wei et al.

ICLR 2024arXiv:2305.18246

#84

Provable and Practical: Efficient Exploration in Reinforcement Learning via Langevin Monte Carlo

Haque Ishfaq, Qingfeng Lan, Pan Xu et al.

33

ICML 2024arXiv:2402.12479

#85

In value-based deep reinforcement learning, a pruned network is a good network

Johan Obando Ceron, Aaron Courville, Pablo Samuel Castro

deep reinforcement learningvalue-based methodssparse trainingnetwork pruning+2

33

NEURIPS 2025arXiv:2505.22019

#86

CPPO: Continual Learning for Reinforcement Learning with Human Feedback

Han Zhang, Yu Lei, Lin Gui et al.

VRAG-RL: Empower Vision-Perception-Based RAG for Visually Rich Information Understanding via Iterative Reasoning with Reinforcement Learning

Qiuchen Wang, Ruixue Ding, Yu Zeng et al.

ICML 2024arXiv:2403.00564

#88

EfficientZero V2: Mastering Discrete and Continuous Control with Limited Data

Shengjie Wang, Shaohuai Liu, Weirui Ye et al.

sample efficiencyreinforcement learningcontinuous controldiscrete control+4

ICML 2025arXiv:2502.00361

#89

Efficient Online Reinforcement Learning for Diffusion Policy

Haitong Ma, Tianyi Chen, Kai Wang et al.

ICML 2024arXiv:2405.17098

#90

Q-value Regularized Transformer for Offline Reinforcement Learning

Shengchao Hu, Ziqing Fan, Chaoqin Huang et al.

offline reinforcement learningconditional sequence modelingdynamic programming methodsaction-value function+4

ICML 2024arXiv:2402.06886

#91

Principled Penalty-based Methods for Bilevel Reinforcement Learning and RLHF

Han Shen, Zhuoran Yang, Tianyi Chen

bilevel optimizationreinforcement learninginverse reinforcement learningrl from human feedback+4

30

ICLR 2025arXiv:2412.07762

#92

Efficient Online Reinforcement Learning Fine-Tuning Need Not Retain Offline Data

Zhiyuan Zhou, Andy Peng, Qiyang Li et al.

reinforcement learning fine-tuningoffline reinforcement learningonline reinforcement learningdistribution mismatch+4

30

NEURIPS 2025arXiv:2507.07969

#93

Reinforcement Learning with Action Chunking

Qiyang Li, Zhiyuan (Paul) Zhou, Sergey Levine

29

ICML 2024arXiv:2402.08848

#94

Hybrid Inverse Reinforcement Learning

Juntao Ren, Gokul Swamy, Steven Wu et al.

inverse reinforcement learningimitation learninghybrid reinforcement learningexpert-competitive rl+4

29

NEURIPS 2025arXiv:2506.07464

#95

DeepVideo-R1: Video Reinforcement Fine-Tuning via Difficulty-aware Regressive GRPO

Jinyoung Park, Jeehye Na, Jinyoung Kim et al.

reinforcement learning fine-tuningvideo large language modelsgroup relative policy optimizationregressive policy optimization+4

28

NEURIPS 2025arXiv:2505.23678

#96

Grounded Reinforcement Learning for Visual Reasoning

Gabriel Sarch, Snigdha Saha, Naitik Khandelwal et al.

28

ICLR 2024arXiv:2404.01220

#97

Entity-Centric Reinforcement Learning for Object Manipulation from Pixels

Dan Haramati, Tal Daniel, Aviv Tamar

ICLR 2024arXiv:2401.11237

#98

Closing the Gap between TD Learning and Supervised Learning - A Generalisation Point of View.

Raj Ghugare, Matthieu Geist, Glen Berseth et al.

ICML 2024arXiv:2402.04764

#99

Code as Reward: Empowering Reinforcement Learning with VLMs

David Venuto, Mohammad Sami Nur Islam, Martin Klissarov et al.

vision-language modelsreinforcement learningreward function generationcode generation+3

ICLR 2024arXiv:2311.12996

#100

RLIF: Interactive Imitation Learning as Reinforcement Learning

Jianlan Luo, Perry Dong, Yuexiang Zhai et al.