Less Experts, Faster Decoding: Cost-Aware Speculative Decoding for Mixture-of-Experts

A new speculative decoding strategy for Mixture-of-Experts models optimizes inference efficiency by accounting for the computational costs of specific expert activations.

Firefly links to the original publisher. The summary above is AI-generated for orientation and may differ from the source. The “current rank score” decays over time so newer significant stories surface first.