Offline Reinforcement Learning

ICML 2024arXiv:2402.00782

#2

Dense Reward for Free in Reinforcement Learning from Human Feedback

Alexander Chan, Hao Sun, Samuel Holt et al.

reinforcement learning from human feedbackreward model attentionpotential-based reward shapingsparse reward problem+3

65

ICML 2024arXiv:2401.16335

#3

Training Diffusion Models Towards Diverse Image Generation with Reinforcement Learning

Zichen Miao, Jiang Wang, Ze Wang et al.

Iterative Data Smoothing: Mitigating Reward Overfitting and Overoptimization in RLHF

Banghua Zhu, Michael Jordan, Jiantao Jiao

reward model learninghuman feedback alignmentreward overoptimizationiterative data smoothing+3

48

ICLR 2024arXiv:2310.19668

#5

DrM: Mastering Visual Reinforcement Learning through Dormant Ratio Minimization

Guowei Xu, Ruijie Zheng, Yongyuan Liang et al.

46

ICLR 2024arXiv:2305.14816

#6

Provable Offline Preference-Based Reinforcement Learning

Wenhao Zhan, Masatoshi Uehara, Nathan Kallus et al.

43

ICML 2024arXiv:2403.00514

#7

Overestimation, Overfitting, and Plasticity in Actor-Critic: the Bitter Lesson of Reinforcement Learning

Michal Nauman, Michał Bortkiewicz, Piotr Milos et al.

off-policy reinforcement learningactor-critic methodsregularization techniquesoverestimation bias+3

41

ICLR 2024arXiv:2309.06599

#8

Reasoning with Latent Diffusion in Offline Reinforcement Learning

Siddarth Venkatraman, Shivesh Khaitan, Ravi Tej Akella et al.

36

ICLR 2025arXiv:2412.07762

#9

Efficient Online Reinforcement Learning Fine-Tuning Need Not Retain Offline Data

Zhiyuan Zhou, Andy Peng, Qiyang Li et al.

reinforcement learning fine-tuningoffline reinforcement learningonline reinforcement learningdistribution mismatch+4

30

ICML 2024arXiv:2402.08848

#10

Hybrid Inverse Reinforcement Learning

Juntao Ren, Gokul Swamy, Steven Wu et al.

inverse reinforcement learningimitation learninghybrid reinforcement learningexpert-competitive rl+4

29

ICLR 2024arXiv:2311.12996

#11

RLIF: Interactive Imitation Learning as Reinforcement Learning

Jianlan Luo, Perry Dong, Yuexiang Zhai et al.

NEURIPS 2025arXiv:2503.22230

#12

Exploring Data Scaling Trends and Effects in Reinforcement Learning from Human Feedback

Wei Shen, Guanlin Liu, Yu Yue et al.

AAAI 2024arXiv:2306.12755

#13

Beyond OOD State Actions: Supported Cross-Domain Offline Reinforcement Learning

Jinxin Liu, Ziqi Zhang, Zhenyu Wei et al.

offline reinforcement learningout-of-distribution state actionscross-domain learningtransition dynamics mismatch+4

ICLR 2024arXiv:2401.11237

#14

Closing the Gap between TD Learning and Supervised Learning - A Generalisation Point of View.

Raj Ghugare, Matthieu Geist, Glen Berseth et al.

ICML 2024arXiv:2312.12275

#15

Emergence of In-Context Reinforcement Learning from Noise Distillation

Ilya Zisman, Vladislav Kurenkov, Alexander Nikulin et al.

in-context reinforcement learningnoise-induced curriculumdata acquisitionsynthetic noise injection+2

26

AAAI 2024arXiv:2210.09241

#16

Reinforcement Learning and Data

Generation for Syntax-Guided Synthesis

offline reinforcement learningdistributional shiftbehavior policy constraintsupport alignment+3

26

ICLR 2024arXiv:2310.12955

#17

Towards Robust Offline Reinforcement Learning under Diverse Data Corruption

Rui Yang, Han Zhong, Jiawei Xu et al.

26

ICLR 2024arXiv:2310.12972

#18

CCIL: Continuity-Based Data Augmentation for Corrective Imitation Learning

Liyiming Ke, Yunchu Zhang, Abhay Deshpande et al.

25

NEURIPS 2025arXiv:2505.20347

#19

SeRL: Self-play Reinforcement Learning for Large Language Models with Limited Data

Wenkai Fang, Shunyu Liu, Yang Zhou et al.

reinforcement learninglarge language modelsself-instruction generationself-rewarding mechanisms+4

25

ICML 2024arXiv:2312.13327

#20

In-Context Reinforcement Learning for Variable Action Spaces

Viacheslav Sinii, Alexander Nikulin, Vladislav Kurenkov et al.

in-context reinforcement learningvariable action spacestransformers pre-trainingmulti-episode contexts+4

25

ICML 2025arXiv:2505.07395

#21

ReinboT: Amplifying Robot Visual-Language Manipulation with Reinforcement Learning

Hongyin Zhang, Zifeng Zhuang, Han Zhao et al.

24

AAAI 2025arXiv:2412.11120

#22

Latent Reward: LLM-Empowered Credit Assignment in Episodic Reinforcement Learning

Yun Qu, Yuhang Jiang, Boyuan Wang et al.

credit assignmentepisodic reinforcement learningreward redistributionlatent reward+3

24

ICLR 2024arXiv:2310.11971

#23

Improving Generalization of Alignment with Human Preferences through Group Invariant Learning

Rui Zheng, Wei Shen, Yuan Hua et al.

23

ICLR 2024arXiv:2312.05742

#24

The Generalization Gap in Offline Reinforcement Learning

Ishita Mediratta, Qingfei You, Minqi Jiang et al.

22

NEURIPS 2025arXiv:2506.05316

#25

Improving Data Efficiency for LLM Reinforcement Fine-tuning Through Difficulty-targeted Online Data Selection and Rollout Replay

Yifan Sun, Jingyan Shen, Yibin Wang et al.

20

NEURIPS 2025arXiv:2502.14819

#26

Learning from Reward-Free Offline Data: A Case for Planning with Latent Dynamics Models

Uladzislau Sobal, Wancong Zhang, Kyunghyun Cho et al.

reward-free offline learninglatent dynamics modelsmodel-based planninggoal-conditioned rl+4

20

ICLR 2024arXiv:2305.14550

#27

When should we prefer Decision Transformers for Offline Reinforcement Learning?

Prajjwal Bhargava, Rohan Chitnis, Alborz Geramifard et al.

AAAI 2024arXiv:2312.00198

#28

Optimal Attack and Defense for Reinforcement Learning

Jeremy McMahan, Young Wu, Xiaojin Zhu et al.

ICLR 2025arXiv:2506.13536

#29

What Matters in Learning from Large-Scale Datasets for Robot Manipulation

Vaibhav Saxena, Matthew Bronars, Nadun Ranawaka Arachchige et al.

imitation learningrobot manipulationdataset compositionpolicy learning+4

NEURIPS 2025arXiv:2506.20520

#30

Asymmetric REINFORCE for off-Policy Reinforcement Learning: Balancing positive and negative rewards

Charles Arnal, Gaëtan Narozniak, Vivien Cabannes et al.

reinforcement learningoff-policy methodspolicy improvement guaranteereward shaping+3

ICML 2024arXiv:2404.18909

#31

Sample-Efficient Robust Multi-Agent Reinforcement Learning in the Face of Environmental Uncertainty

Laixi Shi, Eric Mazumdar, Yuejie Chi et al.

multi-agent reinforcement learningdistributionally robust markov gamesrobust equilibrium strategiesgame-theoretic equilibria+4

NEURIPS 2025arXiv:2203.01707

#32

TESTING STATIONARITY AND CHANGE POINT DETECTION IN REINFORCEMENT LEARNING

Mengbing Li, Chengchun Shi, Zhenke Wu et al.

AAAI 2024arXiv:2312.09539

#33

Situation-Dependent Causal Influence-Based Cooperative Multi-Agent Reinforcement Learning

Xiao Du, Yutong Ye, Pengyu Zhang et al.

AAAI 2024arXiv:2312.15909

#34

Generalizable Task Representation Learning for Offline Meta-Reinforcement Learning with Data Limitations

Renzhe Zhou, Chen-Xiao Gao, Zongzhang Zhang et al.

offline meta-reinforcement learningtask representation learningdata limitationstask auto-encoder+4

AAAI 2025arXiv:2501.06773

#35

Pareto Set Learning for Multi-Objective Reinforcement Learning

Erlong Liu, Yu-Chang Wu, Xiaobin Huang et al.

multi-objective reinforcement learningpareto front learningdecomposition-based methodshypernetwork generation+3

ICLR 2024arXiv:2402.03807

#36

SEABO: A Simple Search-Based Method for Offline Imitation Learning

Jiafei Lyu, Xiaoteng Ma, Le Wan et al.

ICLR 2024arXiv:2310.05422

#37

Reward-Consistent Dynamics Models are Strongly Generalizable for Offline Reinforcement Learning

Fan-Ming Luo, Tian Xu, Xingchen Cao et al.

ICML 2024arXiv:2405.19909

#38

Adaptive Advantage-Guided Policy Regularization for Offline Reinforcement Learning

Tenglong Liu, Yang Li, Yixing Lan et al.

offline reinforcement learningpolicy regularizationout-of-distribution actionsadvantage-guided learning+4

13

ICLR 2024arXiv:2310.07433

#39

Imitation Learning from Observation with Automatic Discount Scheduling

Yuyang Liu, Weijun Dong, Yingdong Hu et al.

13

ICLR 2025arXiv:2406.08973

#40

XLand-100B: A Large-Scale Multi-Task Dataset for In-Context Reinforcement Learning

Alexander Nikulin, Ilya Zisman, Alexey Zemtsov et al.

ICML 2024arXiv:2408.04190

#41

Listwise Reward Estimation for Offline Preference-based Reinforcement Learning

Heewoong Choi, Sangwon Jung, Hongjoon Ahn et al.

preference-based reinforcement learningreward function designhuman feedback learningoffline reinforcement learning+4

ICLR 2024arXiv:2401.08819

#42

Learning from Sparse Offline Datasets via Conservative Density Estimation

Zhepeng Cen, Zuxin Liu, Zitong Wang et al.

ICLR 2025arXiv:2410.08896

#43

MAD-TD: Model-Augmented Data stabilizes High Update Ratio RL

Claas Voelcker, Marcel Hussing, ERIC EATON et al.

ICLR 2025arXiv:2410.03618

#44

Open-World Reinforcement Learning over Long Short-Term Imagination

Jiajian Li, Qi Wang, Yunbo Wang et al.

ICLR 2024arXiv:2310.00771

#45

Pre-training with Synthetic Data Helps Offline Reinforcement Learning

Zecheng Wang, Che Wang, Zixuan Dong et al.

NEURIPS 2025arXiv:2506.00070

#46

Robot-R1: Reinforcement Learning for Enhanced Embodied Reasoning in Robotics

Dongyoung Kim, Huiwon Jang, Sumin Park et al.

reinforcement learningembodied reasoningrobot controlvision-language models+4

ICLR 2024arXiv:2311.15951

#47

Replay across Experiments: A Natural Extension of Off-Policy RL

Dhruva Tirumala, Thomas Lampe, Jose Enrique Chen et al.

ICLR 2024arXiv:2402.12673

#48

Beyond Worst-case Attacks: Robust RL with Adaptive Defense via Non-dominated Policies

Xiangyu Liu, Chenghao Deng, Yanchao Sun et al.

ICLR 2025arXiv:2406.11715

#49

Measuring memorization in RLHF for code completion

Jamie Hayes, I Shumailov, Billy Porter et al.

rlhf alignment processtraining data memorizationdirect preference optimizationpreference learning methods+4

ICML 2024arXiv:2306.01276

#50

Symmetric Replay Training: Enhancing Sample Efficiency in Deep Reinforcement Learning for Combinatorial Optimization

Hyeonah Kim, Minsu Kim, Sungsoo Ahn et al.

deep reinforcement learningcombinatorial optimizationsample efficiencymolecular optimization+3

9

ICML 2024arXiv:2407.12448

#51

Energy-Guided Diffusion Sampling for Offline-to-Online Reinforcement Learning

Xu-Hui Liu, Tian-Shuo Liu, Shengyi Jiang et al.

offline-to-online reinforcement learningdiffusion model samplingenergy-guided samplingdata distribution shift+3

9

ICML 2024arXiv:2402.05876

#52

An Implicit Trust Region Approach to Behavior Regularized Offline Reinforcement Learning

Zhe Zhang, Xiaoyang Tan

Federated Offline Reinforcement Learning: Collaborative Single-Policy Coverage Suffices

Jiin Woo, Laixi Shi, Gauri Joshi et al.

federated reinforcement learningoffline reinforcement learningtabular markov decision processesq-learning algorithms+3

9

ICML 2024arXiv:2406.11905

#54

EvIL: Evolution Strategies for Generalisable Imitation Learning

Silvia Sapora, Gokul Swamy, Christopher Lu et al.

imitation learninginverse reinforcement learningreward model ensemblespolicy transfer+3

ICML 2024arXiv:2405.20984

#55

Bayesian Design Principles for Offline-to-Online Reinforcement Learning

Hao Hu, yiqin yang, Jianing Ye et al.

offline reinforcement learningonline fine-tuningbayesian design principlesprobability-matching agent+4

ICLR 2024arXiv:2406.15042

#56

Behaviour Distillation

Andrei Lupu, Chris Lu, Jarek Liesen et al.

AAAI 2025arXiv:2412.18946

#57

Constraint-Adaptive Policy Switching for Offline Safe Reinforcement Learning

Yassine Chemingui, Aryan Deshwal, Honghao Wei et al.

ICML 2025arXiv:2506.03404

#58

The Impact of On-Policy Parallelized Data Collection on Deep Reinforcement Learning Networks

Walter Mayor, Johan Obando-Ceron, Aaron Courville et al.

7

ICML 2024arXiv:2402.15392

#59

Offline Inverse RL: New Solution Concepts and Provably Efficient Algorithms

Filippo Lazzati, Mirco Mutti, Alberto Maria Metelli

inverse reinforcement learningoffline learningfeasible reward setpessimistic algorithms+3

7

ICLR 2024arXiv:2404.16779

#60

DrS: Learning Reusable Dense Rewards for Multi-Stage Tasks

Tongzhou Mu, Minghua Liu, Hao Su

7

AAAI 2025arXiv:2505.05609

#61

On Corruption-Robustness in Performative Reinforcement Learning

Vasilis Pollatos, Debmalya Mandal, Goran Radanovic

performative reinforcement learningpolicy-dependent environmentscorruption-robustnesshuber contamination model+4

NEURIPS 2025arXiv:2505.18917

#62

Behavior Injection: Preparing Language Models for Reinforcement Learning

Zhepeng Cen, Yihang Yao, William Han et al.

AAAI 2025arXiv:2311.04830

#63

Real-Time Recurrent Reinforcement Learning

Julian Lemmel, Radu Grosu

partially observable mdpsmeta-reinforcement learningtemporal difference learningeligibility traces+4

ICLR 2024arXiv:2310.00036

#64

Cleanba: A Reproducible and Efficient Distributed Reinforcement Learning Platform

Shengyi Huang, Jiayi Weng, Rujikorn Charakorn et al.

ICLR 2025arXiv:2410.10132

#65

Stable Hadamard Memory: Revitalizing Memory-Augmented Agents for Reinforcement Learning

Hung Le, Dung Nguyen, Kien Do et al.

memory-augmented agentspartially observable environmentsreinforcement learninghadamard product+4

AAAI 2025arXiv:2402.05541

#66

FedAA: A Reinforcement Learning Perspective on Adaptive Aggregation for Fair and Robust Federated Learning

Jialuo He, Wei Chen, Xiaojin Zhang

ICLR 2025arXiv:2410.10636

#67

RRL: Recommendation Reverse Learning

Xiaoyu You, Jianwei Xu, Mi Zhang et al.

Adapt-$\infty$: Scalable Continual Multimodal Instruction Tuning via Dynamic Data Selection

Adyasha Maharana, Jaehong Yoon, Tianlong Chen et al.

AAAI 2025arXiv:2402.01886

#69

Inverse Reinforcement Learning by Estimating Expertise of Demonstrators

Mark Beliaev, Ramtin Pedarsani

ICLR 2024arXiv:2310.17786

#70

DUO: Diverse, Uncertain, On-Policy Query Generation and Selection for Reinforcement Learning from Human Feedback

Xuening Feng, Zhaohui Jiang, Timo Kaufmann et al.

Understanding when Dynamics-Invariant Data Augmentations Benefit Model-free Reinforcement Learning Updates

Nicholas Corrado, Josiah Hanna

AAAI 2025arXiv:2412.15429

#72

Offline Safe Reinforcement Learning Using Trajectory Classification

Ze Gong, Akshat Kumar, Pradeep Varakantham

ECCV 2024arXiv:2407.14872

#73

Adapt2Reward: Adapting Video-Language Models to Generalizable Robotic Rewards via Failure Prompts

Yanting Yang, Minghao Chen, Qibo Qiu et al.

vision-language modelslanguage-conditioned rewardrobotic planningreinforcement learning+4

ICML 2025arXiv:2410.14038

#74

Sliding Puzzles Gym: A Scalable Benchmark for State Representation in Visual Reinforcement Learning

Bryan L. M. de Oliveira, Luana G. B. Martins, Bruno Brandão et al.

ICLR 2025arXiv:2405.00746

#75

Leveraging Sub-Optimal Data for Human-in-the-Loop Reinforcement Learning

Calarina Muslimani, Matthew E Taylor

ICLR 2024arXiv:2310.17303

#76

Demonstration-Regularized RL

Daniil Tiapkin, Denis Belomestny, Daniele Calandriello et al.

ICML 2024arXiv:2406.07253

#77

Hybrid Reinforcement Learning from Offline Observation Alone

Yuda Song, J. Bagnell, Aarti Singh

hybrid reinforcement learningoffline observation datareset modeltrace model+4

ICLR 2025arXiv:2410.07933

#78

Offline Hierarchical Reinforcement Learning via Inverse Optimization

Carolin Schmidt, Daniele Gammelli, James Harrison et al.

ICLR 2025arXiv:2407.00699

#79

Model-based Offline Reinforcement Learning with Lower Expectile Q-Learning

Kwanyoung Park, Youngwoon Lee

offline reinforcement learningmodel-based reinforcement learningvalue estimationexpectile regression+3

reward modelinghuman preference learningpolicy optimizationoff-distribution problem+4

#80

Uncertainty and Influence aware Reward Model Refinement for Reinforcement Learning from Human Feedback

Zexu Sun, Yiju Guo, Yankai Lin et al.

ICLR 2025

ICLR 2025arXiv:2410.01954

#81

ComaDICE: Offline Cooperative Multi-Agent Reinforcement Learning with Stationary Distribution Shift Regularization

The Viet Bui, Thanh Nguyen, Tien Mai

offline reinforcement learningmodel-free algorithmsrobustness enhancementdouble-pessimism principle+3

#82

Model-Free Offline Reinforcement Learning with Enhanced Robustness

Chi Zhang, Zain Ulabedeen Farhat, George Atia et al.

ICLR 2025

NEURIPS 2025arXiv:2502.07193

#83

Provably Efficient Online RLHF with One-Pass Reward Modeling

Long-Fei Li, Yu-Yang Qian, Peng Zhao et al.

ICLR 2025arXiv:2502.04141

#84

Behavioral Entropy-Guided Dataset Generation for Offline Reinforcement Learning

Wesley Suttle, Aamodh Suresh, Carlos Nieto-Granda

behavioral entropyoffline reinforcement learningdataset generationstate space exploration+4

ICLR 2025arXiv:2407.04285

#85

Tackling Data Corruption in Offline Reinforcement Learning via Sequence Modeling

Jiawei Xu, Rui Yang, Shuang Qiu et al.

offline reinforcement learningdata corruption robustnesssequence modelingdecision transformer+4

AAAI 2025arXiv:2503.07671

#86

Probabilistic Shielding for Safe Reinforcement Learning

Edwin Hamel-De le Court, Francesco Belardinelli, Alexander W. Goodall

AAAI 2024arXiv:2401.06514

#87

Personalized Reinforcement Learning with a Budget of Policies

Dmitry Ivanov, Omer Ben-Porat

NEURIPS 2025arXiv:2506.08388

#88

Reinforcement Learning Teachers of Test Time Scaling

Edoardo Cetin, Tianyu Zhao, Yujin Tang

ICML 2025arXiv:2502.06861

#89

Visual Reinforcement Learning with Residual Action

Zhenxian Liu, Peixi Peng, Yonghong Tian

Design Considerations in Offline Preference-based RL

Alekh Agarwal, Christoph Dann, Teodor Vanislavov Marinov

ICML 2024arXiv:2402.04493

#91

A Primal-Dual Algorithm for Offline Constrained Reinforcement Learning with Linear MDPs

Kihyuk Hong, Ambuj Tewari

offline reinforcement learninglinear mdpsprimal-dual algorithmconstrained reinforcement learning+3

ICML 2024arXiv:2210.14972

#92

Environment Design for Inverse Reinforcement Learning

Thomas Kleine Buening, Victor Villin, Christos Dimitrakakis

inverse reinforcement learningreward function learningenvironment designsample-efficiency+3

ICML 2025arXiv:2506.13672

#93

The Courage to Stop: Overcoming Sunk Cost Fallacy in Deep Reinforcement Learning

Jiashun Liu, Johan Obando-Ceron, Pablo Samuel Castro et al.

ICML 2025arXiv:2505.19923

#94

Learning to Trust Bellman Updates: Selective State-Adaptive Regularization for Offline RL

Qin-Wen Luo, Ming-Kun Xie, Ye-Wen Wang et al.

ICML 2025arXiv:2404.02113

#95

Position: Lifetime tuning is incompatible with continual reinforcement learning

Golnaz Mesbahi, Parham Mohammad Panahi, Olya Mastikhina et al.

ICML 2024arXiv:2406.06037

#96

POCE: Primal Policy Optimization with Conservative Estimation for Multi-constraint Offline Reinforcement Learning

Jiayi Guan, Li Shen, Ao Zhou et al.

Investigating Pre-Training Objectives for Generalization in Vision-Based Reinforcement Learning

Donghu Kim, Hojoon Lee, Kyungmin Lee et al.

vision-based reinforcement learningpre-training objectivesgeneralization abilityatari benchmark+3

ICML 2025arXiv:2505.16734

#98

Maximum Total Correlation Reinforcement Learning

Bang You, Puze Liu, Huaping Liu et al.

ICML 2025arXiv:2405.17618

#99

Symmetric Reinforcement Learning Loss for Robust Learning on Diverse Tasks and Model Scales

Ju-Seung Byun, Andrew Perrault

ICML 2025arXiv:2507.09177

#100

Continual Reinforcement Learning by Planning with Online World Models

Zichen Liu, Guoji Fu, Chao Du et al.