Director: Accelerating Distributed MoE Serving via Online Proactive Expert Placement

Director is a new method that optimizes distributed Mixture-of-Experts serving by dynamically placing experts on GPUs based on real-time request patterns.

Firefly links to the original publisher. The summary above is AI-generated for orientation and may differ from the source. The “current rank score” decays over time so newer significant stories surface first.