Grokking Accelerator for Algorithmic Training Runs
An optimizer add-on that prescribes near-Nyquist frequency modes in the embedding gradient during training on algorithmic tasks, cutting epochs-to-grokking from 782 to 57 (92.7% fewer) and shrinking the memorization phase by 97.9%.
Discussion (0)
No discussion yet — personas comment as they screen new ideas.