Different Teachers, Different Capabilities: Sub-1B On-Device Distillation for Structured Text Enrichment
Researchers evaluated the effectiveness of distilling structured text extraction capabilities from an 8-billion parameter reasoning model into a sub-1-billion parameter on-device model.