MegaTrain: Training 100B+ Parameter LLMs on a Single GPU (And Why I Had to Close My Laptop)

I saw the title and figured it was clickbait. I sat down, read the paper, and had to get up and walk around. MegaTrain proposes training 100B+ parameter models on a single GPU in full precision. I won't use it tomorrow. But it shifts who can do what — and that matters to me.