"second-order optimization" Papers

11 papers found

Filters:second-order optimization Clear all

Conference

AAAI 2025 (3,028)COLM 2025 (418)CVPR 2025 (2,873)ICCV 2025 (2,701)ICLR 2025 (3,827)ICML 2025 (3,340)ISMAR 2025 (229)NEURIPS 2025 (5,858)AAAI 2024 (2,289)CVPR 2024 (2,716)ECCV 2024 (2,387)ICLR 2024 (2,297)ICML 2024 (2,635)

Paper Type

poster (24,624)paper (8,558)oral (1,594)spotlight (1,421)highlight (975)

A New Perspective on Shampoo's Preconditioner

Depen Morwani, Itai Shapira, Nikhil Vyas et al.

ICLR 2025arXiv:2406.17748

An Illustrated Guide to Automatic Sparse Differentiation

Adrian Hill, Guillaume Dalle, Alexis Montoison

Debiasing Mini-Batch Quadratics for Applications in Deep Learning

Lukas Nicola Tatzel, Bálint Mucsányi, Osane Hackel et al.

ICLR 2025arXiv:2410.14325

Gradient Alignment in Physics-informed Neural Networks: A Second-Order Optimization Perspective

Sifan Wang, Ananyae bhartari, Bowen Li et al.

NEURIPS 2025arXiv:2502.00604

How to Scale Second-Order Optimization

Charlie Chen, Shikai Qiu, Hoang Phan et al.

Newton Meets Marchenko-Pastur: Massively Parallel Second-Order Optimization with Hessian Sketching and Debiasing

Elad Romanov, Fangzhao Zhang, Mert Pilanci

ICLR 2025arXiv:2410.01374

Train with Perturbation, Infer after Merging: A Two-Stage Framework for Continual Learning

Haomiao Qiu, Miao Zhang, Ziyue Qiao et al.

NEURIPS 2025arXiv:2505.22389

Can We Remove the Square-Root in Adaptive Gradient Methods? A Second-Order Perspective

Wu Lin, Felix Dangel, Runa Eschenhagen et al.

ICML 2024arXiv:2402.03496

Error Feedback Can Accurately Compress Preconditioners

Ionut-Vlad Modoranu, Aleksei Kalinov, Eldar Kurtic et al.

ICML 2024arXiv:2306.06098

Revisiting Scalable Hessian Diagonal Approximations for Applications in Reinforcement Learning

Mohamed Elsayed, Homayoon Farrahi, Felix Dangel et al.

ICML 2024arXiv:2406.03276

Structured Inverse-Free Natural Gradient Descent: Memory-Efficient & Numerically-Stable KFAC

Wu Lin, Felix Dangel, Runa Eschenhagen et al.