Efficient Inference

inference scaling lawscompute-optimal inferencelarge language modelstest-time scaling+4

#2

Inference Scaling Laws: An Empirical Analysis of Compute-Optimal Inference for LLM Problem-Solving

Yangzhen Wu, Zhiqing Sun, Shanda Li et al.

ICLR 2025

146

AAAI 2025arXiv:2403.14520

#3

Cobra: Extending Mamba to Multi-Modal Large Language Model for Efficient Inference

Han Zhao, Min Zhang, Wei Zhao et al.

multimodal large language modelslinear computational complexitymamba language modelmodal fusion schemes+4

110

ICML 2024arXiv:2404.09636

#4

All-in-one simulation-based inference

Manuel Gloeckler, Michael Deistler, Christian Weilbach et al.

amortized bayesian inferencesimulation-based inferenceprobabilistic diffusion modelstransformer architectures+3

62

NEURIPS 2025arXiv:2502.00234

#5

Fast Solvers for Discrete Diffusion Models: Theory and Applications of High-Order Algorithms

Yinuo Ren, Haoxuan Chen, Yuchen Zhu et al.

discrete diffusion modelshigh-order algorithmsnumerical inference schemesgenerative modeling+4

32

ICML 2025arXiv:2501.17974

#6

Think Smarter not Harder: Adaptive Reasoning with Inference Aware Optimization

Zishun Yu, Tengyu Xu, Di Jin et al.

25

ICML 2025arXiv:2503.19595

#7

Optimizing Language Models for Inference Time Objectives using Reinforcement Learning

Yunhao Tang, Kunhao Zheng, Gabriel Synnaeve et al.

23

ICML 2025arXiv:2410.16377

#8

A Simple Model of Inference Scaling Laws

Noam Levi

23

ICLR 2024arXiv:2310.09163

#9

Jointly-Learned Exit and Inference for a Dynamic Neural Network

Florence Regol, Joud Chataoui, Mark Coates

16

NEURIPS 2025arXiv:2502.07191

#10

Bag of Tricks for Inference-time Computation of LLM Reasoning

Fan LIU, Wen-Shuo Chao, Naiqiang Tan et al.

inference-time computationreasoning tasksproposer-verifier pipelinecandidate solution generation+3

12

CVPR 2024arXiv:2405.00984

#11

FREE: Faster and Better Data-Free Meta-Learning

Yongxian Wei, Zixuan Hu, Zhenyi Wang et al.

9

ICLR 2024arXiv:2107.03759

#12

Analytically Tractable Hidden-States Inference in Bayesian Neural Networks

Luong-Ha Nguyen, James-A. Goulet

8

CVPR 2025arXiv:2412.15341

#13

Efficient Fine-Tuning and Concept Suppression for Pruned Diffusion Models

Reza Shirkavand, Peiran Yu, Shangqian Gao et al.

8

ICLR 2025arXiv:2405.14105

#14

Distributed Speculative Inference (DSI): Speculation Parallelism for Provably Faster Lossless Language Model Inference

Nadav Timor, Jonathan Mamou, Daniel Korat et al.

speculative inferencelossless language model inferencedistributed inferencespeculation parallelism+4

7

NEURIPS 2025arXiv:2511.05664

#15

KLASS: KL-Guided Fast Inference in Masked Diffusion Models

Seo Hyun Kim, Sunwoo Hong, Hojung Jung et al.

masked diffusion modelsfast inference samplingkl divergence guidancetoken-level stability+4

6

NEURIPS 2025arXiv:2410.19315

#16

Brain-like Variational Inference

Hadi Vafaii, Dekel Galor, Jacob Yates

variational inferenceevidence lower boundvariational autoencoderspiking neural networks+4

5

ICML 2025arXiv:2505.05950

#17

FloE: On-the-Fly MoE Inference on Memory-constrained GPU

Yuxin Zhou, Zheng Li, Jun Zhang et al.

5

NEURIPS 2025arXiv:2504.17660

#18

Effortless, Simulation-Efficient Bayesian Inference using Tabular Foundation Models

Julius Vetter, Manuel Gloeckler, Daniel Gedon et al.

simulation-based inferencebayesian inferencetabular foundation modelsneural posterior estimation+4

4

ICCV 2025arXiv:2506.07725

#19

ETA: Efficiency through Thinking Ahead, A Dual Approach to Self-Driving with Large Models

Shadi Hamdan, Chonghao Sima, Zetong Yang et al.

dual-system architectureself-driving systemsasynchronous systemaction mask mechanism+4

4

ICLR 2025arXiv:2406.04508

#20

OCCAM: Towards Cost-Efficient and Accuracy-Aware Classification Inference

Dujian Ding, Bicheng Xu, Laks Lakshmanan

3

ICLR 2025arXiv:2410.12982

#21

Flash Inference: Near Linear Time Inference for Long Convolution Sequence Models and Beyond

Costin-Andrei Oncescu, Sanket Jayant Purandare, Stratos Idreos et al.

long convolution sequence modelssubquadratic architecturesattention mechanism alternativessequence generative models+4

2

ICML 2025arXiv:2411.06360

#22

An Efficient Matrix Multiplication Algorithm for Accelerating Inference in Binary and Ternary Neural Networks

Mohsen Dehghankar, Mahdi Erfanian, Abolfazl Asudeh

1

COLM 2025arXiv:2410.21340

#23

Meta-Learning for Speeding Up Large Model Inference in Decentralized Environments

Yipeng Du, Zihao Wang, Ahmad Farhan et al.

Fast inferenceLLMsmeta learningdecentralized systems

1

COLM 2025arXiv:2504.07247

#24

Resource-efficient Inference with Foundation Model Programs

Lunyiu Nie, Zhimin Ding, Kevin Yu et al.

Neurosymblic ProgrammingMultimodal LMsAgent ProgrammingLLM Computational Efficiency+1

1

graph contrastive learningfast inferencemessage passingmlp-gnn coupling+3

#25

An Efficient Subgraph-Inferring Framework for Large-Scale Heterogeneous Graphs

Wei Zhou, Hong Huang, Ruize Shi et al.

Efficient Contrastive Learning for Fast and Accurate Inference on Graphs

Teng Xiao, Huaisheng Zhu, Zhiwei Zhang et al.

ICML 2024

ICCV 2025arXiv:2508.09262

#27

Harnessing Input-Adaptive Inference for Efficient VLN

Dongwoo Kang, Akhil Perincherry, Zachary Coalson et al.

vision-and-language navigationmulti-modal transformer modelsinput-adaptive inferenceearly-exit methods+3

ICML 2024arXiv:2405.04513

#28

Switchable Decision: Dynamic Neural Generation Networks

Shujian Zhang, Korawat Tanwisuth, Chengyue Gong et al.

auto-regressive generation modelsinference accelerationdynamic computation allocationconstrained optimization+4

#29

Gaussian Ensemble Belief Propagation for Efficient Inference in High-Dimensional, Black-box Systems

Dan MacKinlay, Russell Tsuchida, Daniel Pagendam et al.

Towards Efficient Low-Order Hybrid Optimizer for Language Model Fine-Tuning

Minping Chen, You-Liang Huang, Zeyi Wen

AAAI 2025