Don't Ignore the Tail: Decoupling top-K Probabilities for Efficient Language Model Distillation
arXiv:2602.20816v1 Announce Type: new Abstract: The core learning signal used in language model distillation is the standard Kullback-Leibler (KL) divergence between the student and teacher …
Sayantan Dasgupta, Trevor Cohn, Timothy Baldwin
조회수 68회