A Unified Approach to Interpreting Knowledge Distillation for Large Language Models via Interactions

This paper proposes a unified approach to explain the mechanisms of knowledge distillation in large language models by decomposing output scores into multi-variable interactions.

Firefly links to the original publisher. The summary above is AI-generated for orientation and may differ from the source. The “current rank score” decays over time so newer significant stories surface first.