"adam optimizer" Papers
4 papers found
Conference
Deconstructing What Makes a Good Optimizer for Autoregressive Language Models
Rosie Zhao, Depen Morwani, David Brandfonbrener et al.
ICLR 2025
37
citations
DP-AdamBC: Your DP-Adam Is Actually DP-SGD (Unless You Apply Bias Correction)
Qiaoyue Tang, Frederick Shpilevskiy, Mathias Lécuyer
AAAI 2024paperarXiv:2312.14334
30
citations
Studying K-FAC Heuristics by Viewing Adam through a Second-Order Lens
Ross Clarke, Jose Miguel Hernandez-Lobato
ICML 2024arXiv:2310.14963
2
citations
Understanding Adam Optimizer via Online Learning of Updates: Adam is FTRL in Disguise
Kwangjun Ahn, Zhiyu Zhang, Yunbum Kook et al.
ICML 2024arXiv:2402.01567
22
citations