Running on CPU Upgrade 748 The Ultimate MoE Training Handbook 🧩 748 A hands-on guide to training Mixture-of-Experts models
Symmetry-Compatible Principle for Optimizer Design: Embeddings, LM Heads, SwiGLU MLPs, and MoE Routers Paper • 2605.18106 • Published May 18 • 2
Symmetry-Compatible Principle for Optimizer Design: Embeddings, LM Heads, SwiGLU MLPs, and MoE Routers Paper • 2605.18106 • Published May 18 • 2
METAGENE-1: Metagenomic Foundation Model for Pandemic Monitoring Paper • 2501.02045 • Published Jan 3, 2025 • 23