Category

Academic

Academic · 1 min

SODA: Semi On-Policy Black-Box Distillation for Large Language Models

arXiv:2604.03873v1 Announce Type: new Abstract: Black-box knowledge distillation for large language models presents a strict trade-off. Simple off-policy methods (e.g., sequence-level knowledge distillation) struggle to …

Xiwen Chen, Jingjing Wang, Wenhui Zhu, Peijie Qiu, Xuanzhao Dong, Hejian Sang, Zhipeng Wang, Alborz Geramifard, Feng Luo
49 views
Academic · 1 min

A Bayesian Information-Theoretic Approach to Data Attribution

arXiv:2604.03858v1 Announce Type: new Abstract: Training Data Attribution (TDA) seeks to trace model predictions back to influential training examples, enhancing interpretability and safety. We formulate …

Dharmesh Tailor, Nicol\`o Felicioni, Kamil Ciosek
18 views