Non-Convex Optimization

On Penalty Methods for Nonconvex Bilevel Optimization and First-Order Stochastic Approximation

Jeongyeol Kwon, Dohyun Kwon, Stephen Wright et al.

Gradient Alignment in Physics-informed Neural Networks: A Second-Order Optimization Perspective

Sifan Wang, Ananyae bhartari, Bowen Li et al.

Smooth Tchebycheff Scalarization for Multi-Objective Optimization

Xi Lin, Xiaoyuan Zhang, Zhiyuan Yang et al.

Methods for Convex $(L_0,L_1)$-Smooth Optimization: Clipping, Acceleration, and Adaptivity

Eduard Gorbunov, Nazarii Tupitsa, Sayantan Choudhury et al.

Optimizing $(L_0, L_1)$-Smooth Functions by Gradient Methods

Daniil Vankov, Anton Rodomanov, Angelia Nedich et al.

The Loss Landscape of Deep Linear Neural Networks: a Second-order Analysis

El Mehdi Achour, Francois Malgouyres, Sebastien Gerchinovitz

Moreau Envelope for Nonconvex Bi-Level Optimization: A Single-Loop and Hessian-Free Solution Strategy

Risheng Liu, Zhu Liu, Wei Yao et al.

Leaving the Nest: Going beyond Local Loss Functions for Predict-Then-Optimize

Sanket Shah, Bryan Wilder, Andrew Perrault et al.

Position: Optimization in SciML Should Employ the Function Space Geometry

Johannes Müller, Marius Zeinhofer

Optimal Hessian/Jacobian-Free Nonconvex-PL Bilevel Optimization

Feihu Huang

Preference Optimization on Pareto Sets: On a Theory of Multi-Objective Optimization

Abhishek Roy, Geelon So, Yian Ma

How to Make the Gradients Small Privately: Improved Rates for Differentially Private Non-Convex Optimization

Andrew Lowy, Jonathan Ullman, Stephen Wright

Near-Optimal Solutions of Constrained Learning Problems

Juan Elenter, Luiz Chamon, Alejandro Ribeiro

Decentralized Gradient-Free Methods for Stochastic Non-smooth Non-convex Optimization

Zhenwei Lin, Jingfan Xia, Qi Deng et al.

Private Zeroth-Order Nonsmooth Nonconvex Optimization

Qinzi Zhang, Hoang Tran, Ashok Cutkosky

No Wrong Turns: The Simple Geometry Of Neural Networks Optimization Paths

Charles Guille-Escuret, Hiroki Naganuma, Kilian Fatras et al.

Convex Relaxations of ReLU Neural Networks Approximate Global Optima in Polynomial Time

Sungyoon Kim, Mert Pilanci

Boundary Exploration for Bayesian Optimization With Unknown Physical Constraints

Yunsheng Tian, Ane Zuniga, Xinwei Zhang et al.

Stochastic Weakly Convex Optimization beyond Lipschitz Continuity

Wenzhi Gao, Qi Deng

General framework for online-to-nonconvex conversion: Schedule-free SGD is also effective for nonconvex optimization

Kwangjun Ahn, Gagik Magakyan, Ashok Cutkosky

Locally Convex Global Loss Network for Decision-Focused Learning

Haeun Jeon, Hyunglip Bae, Minsu Park et al.

Nesterov acceleration in benignly non-convex landscapes

Kanan Gupta, Stephan Wojtowytsch

Improved Rates of Differentially Private Nonconvex-Strongly-Concave Minimax Optimization

Ruijia Zhang, Mingxi Lei, Meng Ding et al.

Optimization by Parallel Quasi-Quantum Annealing with Gradient-Based Sampling

Yuma Ichikawa, Yamato Arai

The Global Convergence Time of Stochastic Gradient Descent in Non-Convex Landscapes: Sharp Estimates via Large Deviations

Waïss Azizian, Franck Iutzeler, Jérôme Malick et al.

Zeroth-Order Optimization Finds Flat Minima

Liang Zhang, Bingcong Li, Kiran Thekumparampil et al.

Chasing Convex Functions with Long-term Constraints

Adam Lechowicz, Nicolas Christianson, Bo Sun et al.

Beyond Value Functions: Single-Loop Bilevel Optimization under Flatness Conditions

Liuyuan Jiang, Quan Xiao, Lisha Chen et al.

Revisiting Frank-Wolfe for Structured Nonconvex Optimization

Hoomaan Maskan, Yikun Hou, Suvrit Sra et al.

From Inverse Optimization to Feasibility to ERM

Saurabh Mishra, Anant Raj, Sharan Vaswani

Aligned Multi Objective Optimization

Yonathan Efroni, Ben Kretzu, Daniel Jiang et al.

On Linear Convergence in Smooth Convex-Concave Bilinearly-Coupled Saddle-Point Optimization: Lower Bounds and Optimal Algorithms

Ekaterina Borodich, Alexander Gasnikov, Dmitry Kovalev

Projection-Free Variance Reduction Methods for Stochastic Constrained Multi-Level Compositional Optimization

Wei Jiang, Sifan Yang, Wenhao Yang et al.

Exploring The Loss Landscape Of Regularized Neural Networks Via Convex Duality

Sungyoon Kim, Aaron Mishkin, Mert Pilanci

Improving Pareto Set Learning for Expensive Multi-objective Optimization via Stein Variational Hypernetworks

Minh-Duc Nguyen, Phuong Mai Dinh, Quang-Huy Nguyen et al.

Inverse Optimization via Learning Feasible Regions

Ke Ren, Peyman Mohajerin Esfahani, Angelos Georghiou

Stochastic Smoothed Primal-Dual Algorithms for Nonconvex Optimization with Linear Inequality Constraints

Ruichuan Huang, Jiawei Zhang, Ahmet Alacaoglu

Non-stationary Bandit Convex Optimization: A Comprehensive Study

Xiaoqi Liu, Dorian Baudry, Julian Zimmert et al.

Solver-Free Decision-Focused Learning for Linear Optimization Problems

Senne Berden, Ali Mahmutoğulları, Dimos Tsouros et al.

Compact Optimality Verification for Optimization Proxies

Wenbo Chen, Haoruo Zhao, Mathieu Tanneau et al.

Stochastic Gradients under Nuisances

Facheng Yu, Ronak Mehta, Alex Luedtke et al.

Convex Formulations for Training Two-Layer ReLU Neural Networks

Karthik Prakhya, Tolga Birdal, Alp Yurtsever

GSO-Net: Grid Surface Optimization via Learning Geometric Constraints

Chaoyun Wang, Jingmin Xin, Nanning Zheng et al.

A Regularized Newton Method for Nonconvex Optimization with Global and Local Complexity Guarantees

Yuhao Zhou, Jintao Xu, Bingrui Li et al.

Balancing Gradient and Hessian Queries in Non-Convex Optimization

Deeksha Adil, Brian Bullins, Aaron Sidford et al.

Sample-and-Bound for Non-convex Optimization

Yaoguang Zhai, Zhizhen Qin, Sicun Gao

Exploring Landscapes for Better Minima along Valleys

Tong Zhao, Jiacheng Li, Yuanchang Zhou et al.

Inexact Newton-type Methods for Optimisation with Nonnegativity Constraints

Oscar Smee, Fred Roosta

Efficient Optimization with Orthogonality Constraint: a Randomized Riemannian Submanifold Method

Andi Han, Pierre-Louis Poirion, Akiko Takeda

Differentiable Convex Polyhedra Optimization from Multi-view Images

Daxuan Ren, Haiyi Mei, Hezi Shi et al.

Langevin Multiplicative Weights Update with Applications in Polynomial Portfolio Management

Yi Feng, Xiao Wang, Tian Xie

Generalized Smooth Bilevel Optimization with Nonconvex Lower-Level

Siqi Zhang, Xing Huang, Feihu Huang

A Near-Optimal Algorithm for Decentralized Convex-Concave Finite-Sum Minimax Optimization

Hongxu Chen, Ke Wei, Haishan Ye et al.

Reducing Variance of Stochastic Optimization for Approximating Nash Equilibria in Normal-Form Games

Linjian Meng, Wubing Chen, Wenbin Li et al.

Optimization over Sparse Support-Preserving Sets: Two-Step Projection with Global Optimality Guarantees

William de Vazelhes, Xiaotong Yuan, Bin Gu

TSP: A Two-Sided Smoothed Primal-Dual Method for Nonconvex Bilevel Optimization

Songtao Lu

A Generalization Result for Convergence in Learning-to-Optimize

Michael Sucker, Peter Ochs

High-Probability Bound for Non-Smooth Non-Convex Stochastic Optimization with Heavy Tails

Langqi Liu, Yibo Wang, Lijun Zhang

Barrier Algorithms for Constrained Non-Convex Optimization

Pavel Dvurechenskii, Mathias Staudigl

Multi-View Randomized Kernel Classification via Nonconvex Optimization

Zeroth-Order Methods for Constrained Nonconvex Nonsmooth Stochastic Optimization

Zhuanghua Liu, Cheng Chen, Luo Luo et al.

Decentralized Sum-of-Nonconvex Optimization

Zhuanghua Liu, Bryan Kian Hsiang Low

Fast Non-Log-Concave Sampling under Nonconvex Equality and Inequality Constraints with Landing

Kijung Jeon, Michael Muehlebach, Molei Tao

Distributed Retraction-Free and Communication-Efficient Optimization on the Stiefel Manifold

Yilong Song, Peijin Li, Bin Gao et al.

Stochastic Momentum Methods for Non-smooth Non-Convex Finite-Sum Coupled Compositional Optimization

Xingyu Chen, Bokun Wang, Ming Yang et al.

No Internal Regret with Non-convex Loss Functions

Solving hidden monotone variational inequalities with surrogate losses

Ryan D'Orazio, Danilo Vucetic, Zichu Liu et al.

Global Optimality for Non-linear Constrained Restoration Problems via Invexity

Samuel Pinilla, Jeyan Thiyagalingam

Semi-infinite Nonconvex Constrained Min-Max Optimization

Cody Melcher, Zeinab Alizadeh, Lindsey Hiett et al.

Accelerating Cutting-Plane Algorithms via Reinforcement Learning Surrogates

12620 Kyle Mana, Fernando Acero, Stephen Mak et al.

Safe Online Convex Optimization with Heavy-Tailed Observation Noises

Yunhao Yang, Bo Xue, Yunzhi Hao et al.

Descent with Misaligned Gradients and Applications to Hidden Convexity

Aditya Bhaskara, Ashok Cutkosky, Ravi Kumar et al.

Sharper Convergence Rates for Nonconvex Optimisation via Reduction Mappings

Evan Markou, Thalaiyasingam Ajanthan, Stephen Gould

Local Composite Saddle Point Optimization

Site Bai, Brian Bullins

Boundary Decomposition for Finding Nadir Objective Vector in Multi-Objective Discrete Optimization

Ruihao Zheng, Zhenkun Wang

Leveraging augmented-Lagrangian techniques for differentiating over infeasible quadratic programs in machine learning

Antoine Bambade, Fabian Schramm, Adrien Taylor et al.

Finite-Time Analysis of Stochastic Nonconvex Nonsmooth Optimization on the Riemannian Manifolds

Emre Sahinoglu, Youbang Sun, Shahin Shahrampour

Accelerated Distance-adaptive Methods for Hölder Smooth and Convex Optimization

Yijin Ren, Haifeng Xu, Qi Deng

Fast Projection-Free Approach (without Optimization Oracle) for Optimization over Compact Convex Set

Chenghao Liu, Enming Liang, Minghua Chen

Overcoming the Optimizer's Curse: Obtaining Realistic Prescriptions from Neural Networks

Asterios Tsiourvas, Georgia Perakis