"zeroth-order optimization" Papers
23 papers found
Conference
Addax: Utilizing Zeroth-Order Gradients to Improve Memory Efficiency and Performance of SGD for Fine-Tuning Language Models
Zeman Li, Xinwei Zhang, Peilin Zhong et al.
Collaborative Discrete-Continuous Black-Box Prompt Learning for Language Models
Hualin Zhang, Haozhen Zhang, Zhekai Liu et al.
CONGO: Compressive Online Gradient Optimization
Jeremy Carleton, Prathik Vijaykumar, Divyanshu Saxena et al.
Hybrid Decentralized Optimization: Leveraging Both First- and Zeroth-Order Optimizers for Faster Convergence
Shayan Talaei, Matin Ansaripour, Giorgi Nadiradze et al.
On the Optimal Construction of Unbiased Gradient Estimators for Zeroth-Order Optimization
Shaocong Ma, Heng Huang
PseuZO: Pseudo-Zeroth-Order Algorithm for Training Deep Neural Networks
Pengyun Yue, Xuanlin Yang, Mingqing Xiao et al.
Revisiting Zeroth-Order Optimization: Minimum-Variance Two-Point Estimators and Directionally Aligned Perturbations
Shaocong Ma, Heng Huang
SECA: Semantically Equivalent and Coherent Attacks for Eliciting LLM Hallucinations
Buyun Liang, Liangzu Peng, Jinqi Luo et al.
Second-Order Fine-Tuning without Pain for LLMs: A Hessian Informed Zeroth-Order Optimizer
Yanjun Zhao, Sizhe Dang, Haishan Ye et al.
SharpZO: Hybrid Sharpness-Aware Vision Language Model Prompt Tuning via Forward-Only Passes
Yifan Yang, Zhen Zhang, Rupak Vignesh Swaminathan et al.
Sparse MeZO: Less Parameters for Better Performance in Zeroth-Order LLM Fine-Tuning
Yong Liu, Zirui Zhu, Chaoyu Gong et al.
Stochastic Regret Guarantees for Online Zeroth- and First-Order Bilevel Optimization
Parvin Nazari, Bojian Hou, Davoud Ataee Tarzanagh et al.
Double Variance Reduction: A Smoothing Trick for Composite Optimization Problems without First-Order Gradient
Hao Di, Haishan Ye, Yueling Zhang et al.
Federated Full-Parameter Tuning of Billion-Sized Language Models with Communication Cost under 18 Kilobytes
Zhen Qin, Daoyuan Chen, Bingchen Qian et al.
Gradient Compressed Sensing: A Query-Efficient Gradient Estimator for High-Dimensional Zeroth-Order Optimization
Ruizhong Qiu, Hanghang Tong
Improved Dimensionality Dependence for Zeroth-Order Optimisation over Cross-Polytopes
Weijia Shao
Performative Prediction with Bandit Feedback: Learning through Reparameterization
Yatong Chen, Wei Tang, Chien-Ju Ho et al.
Quantum Algorithm for Online Exp-concave Optimization
Jianhao He, Chengchang Liu, Xutong Liu et al.
Revisiting Zeroth-Order Optimization for Memory-Efficient LLM Fine-Tuning: A Benchmark
Yihua Zhang, Pingzhi Li, Junyuan Hong et al.
Riemannian Accelerated Zeroth-order Algorithm: Improved Robustness and Lower Query Complexity
Chang He, Zhaoye Pan, Xiao Wang et al.
Variance-reduced Zeroth-Order Methods for Fine-Tuning Language Models
Tanmay Gautam, Youngsuk Park, Hao Zhou et al.
Zeroth-Order Methods for Constrained Nonconvex Nonsmooth Stochastic Optimization
Zhuanghua Liu, Cheng Chen, Luo Luo et al.
ZO-AdaMU Optimizer: Adapting Perturbation by the Momentum and Uncertainty in Zeroth-Order Optimization
Shuoran Jiang, Qingcai Chen, Yang Xiang et al.