"self-distillation" Papers
21 papers found
Conference
Beyond Autoregression: Fast LLMs via Self-Distillation Through Time
Justin Deschenaux, Caglar Gulcehre
COIN: Confidence Score-Guided Distillation for Annotation-Free Cell Segmentation
Sanghyun Jo, Seo Lee, Seungwoo Lee et al.
DebGCD: Debiased Learning with Distribution Guidance for Generalized Category Discovery
Yuanpei Liu, Kai Han
FedLPA: Local Prior Alignment for Heterogeneous Federated Generalized Category Discovery
Geeho Kim, Jinu Lee, Bohyung Han
How to build a consistency model: Learning flow maps via self-distillation
Nicholas Boffi, Michael Albergo, Eric Vanden-Eijnden
LoftUp: Learning a Coordinate-Based Feature Upsampler for Vision Foundation Models
Haiwen Huang, Anpei Chen, Volodymyr Havrylov et al.
Long-tailed Adversarial Training with Self-Distillation
Seungju Cho, Hongsin Lee, Changick Kim
MAESTRO: Masked Encoding Set Transformer with Self-Distillation
Matthew Lee, Jaesik Kim, Matei Ionita et al.
Rethinking Self-Distillation: Label Averaging and Enhanced Soft Label Refinement with Partial Labels
Hyeonsu Jeong, Hye Won Chung
SeerAttention: Self-distilled Attention Gating for Efficient Long-context Prefilling
Yizhao Gao, Zhichen Zeng, DaYou Du et al.
Shortcutting Pre-trained Flow Matching Diffusion Models is Almost Free Lunch
Xu Cai, Yang Wu, Qianli Chen et al.
Synergy Between the Strong and the Weak: Spiking Neural Networks are Inherently Self-Distillers
Yongqi Ding, Lin Zuo, Mengmeng Jing et al.
The Effect of Optimal Self-Distillation in Noisy Gaussian Mixture Model
Kaito Takanami, Takashi Takahashi, Ayaka Sakata
Unbiased Region-Language Alignment for Open-Vocabulary Dense Prediction
Yunheng Li, Yuxuan Li, Quan-Sheng Zeng et al.
Vision Transformers with Self-Distilled Registers
Zipeng Yan, Yinjie Chen, Chong Zhou et al.
Diffusion Models for Monocular Depth Estimation: Overcoming Challenging Conditions
Fabio Tosi, Pierluigi Zama Ramirez, Matteo Poggi
Enhanced Sparsification via Stimulative Training
Shengji Tang, Weihao Lin, Hancheng Ye et al.
EPSD: Early Pruning with Self-Distillation for Efficient Model Compression
Dong Chen, Ning Liu, Yichen Zhu et al.
Open Vocabulary 3D Scene Understanding via Geometry Guided Self-Distillation
Pengfei Wang, Yuxi Wang, Shuai Li et al.
Self-Distillation Regularized Connectionist Temporal Classification Loss for Text Recognition: A Simple Yet Effective Approach
Ziyin Zhang, Ning Lu, Minghui Liao et al.
SILC: Improving Vision Language Pretraining with Self-Distillation
Muhammad Ferjad Naeem, Yongqin Xian, Xiaohua Zhai et al.