Gefen: Optimized Stochastic Optimizer

The proposed Gefen optimizer reduces the memory footprint of large-scale model training by sharing second-moment estimates and quantizing first-moment states.

Firefly links to the original publisher. The summary above is AI-generated for orientation and may differ from the source. The “current rank score” decays over time so newer significant stories surface first.