Academic

Academic

Academic · 약 1분

Fast and Effective On-policy Distillation from Reasoning Prefixes

arXiv:2602.15260v1 Announce Type: new Abstract: On-policy distillation (OPD), which samples trajectories from the student model and supervises them with a teacher at the token level, …

Dongxu Zhang, Zhichao Yang, Sepehr Janghorbani, Jun Han, Andrew Ressler II, Qian Qian, Gregory D. Lyng, Sanjit Singh Batra, Robert E. Tillman
조회수 61회
Academic · 약 1분

On Surprising Effectiveness of Masking Updates in Adaptive Optimizers

arXiv:2602.15322v1 Announce Type: new Abstract: Training large language models (LLMs) relies almost exclusively on dense adaptive optimizers with increasingly sophisticated preconditioners. We challenge this by …

Taejong Joo, Wenhan Xia, Cheolmin Kim, Ming Zhang, Eugene Ie
조회수 23회
Academic · 약 1분

Fractional-Order Federated Learning

arXiv:2602.15380v1 Announce Type: new Abstract: Federated learning (FL) allows remote clients to train a global model collaboratively while protecting client privacy. Despite its privacy-preserving benefits, …

Mohammad Partohaghighi, Roummel Marcia, YangQuan Chen
조회수 38회
Academic · 약 1분

Doubly Stochastic Mean-Shift Clustering

arXiv:2602.15393v1 Announce Type: new Abstract: Standard Mean-Shift algorithms are notoriously sensitive to the bandwidth hyperparameter, particularly in data-scarce regimes where fixed-scale density estimation leads to …

Tom Trigano, Yann Sepulcre, Itshak Lapidot
조회수 38회