Poster "second-order optimization" Papers
11 papers found
Conference
A New Perspective on Shampoo's Preconditioner
Depen Morwani, Itai Shapira, Nikhil Vyas et al.
ICLR 2025arXiv:2406.17748
35
citations
An Illustrated Guide to Automatic Sparse Differentiation
Adrian Hill, Guillaume Dalle, Alexis Montoison
ICLR 2025
Debiasing Mini-Batch Quadratics for Applications in Deep Learning
Lukas Nicola Tatzel, Bálint Mucsányi, Osane Hackel et al.
ICLR 2025arXiv:2410.14325
2
citations
Gradient Alignment in Physics-informed Neural Networks: A Second-Order Optimization Perspective
Sifan Wang, Ananyae bhartari, Bowen Li et al.
NEURIPS 2025arXiv:2502.00604
38
citations
How to Scale Second-Order Optimization
Charlie Chen, Shikai Qiu, Hoang Phan et al.
NEURIPS 2025
Newton Meets Marchenko-Pastur: Massively Parallel Second-Order Optimization with Hessian Sketching and Debiasing
Elad Romanov, Fangzhao Zhang, Mert Pilanci
ICLR 2025arXiv:2410.01374
2
citations
Train with Perturbation, Infer after Merging: A Two-Stage Framework for Continual Learning
Haomiao Qiu, Miao Zhang, Ziyue Qiao et al.
NEURIPS 2025arXiv:2505.22389
Can We Remove the Square-Root in Adaptive Gradient Methods? A Second-Order Perspective
Wu Lin, Felix Dangel, Runa Eschenhagen et al.
ICML 2024arXiv:2402.03496
19
citations
Error Feedback Can Accurately Compress Preconditioners
Ionut-Vlad Modoranu, Aleksei Kalinov, Eldar Kurtic et al.
ICML 2024arXiv:2306.06098
6
citations
Revisiting Scalable Hessian Diagonal Approximations for Applications in Reinforcement Learning
Mohamed Elsayed, Homayoon Farrahi, Felix Dangel et al.
ICML 2024arXiv:2406.03276
7
citations
Structured Inverse-Free Natural Gradient Descent: Memory-Efficient & Numerically-Stable KFAC
Wu Lin, Felix Dangel, Runa Eschenhagen et al.
ICML 2024