Collection of stuff I've written about that.
November 17, 2024
This blog will walk through the motions of rebuilding the ZeRO optimizer and FSDP from scratch.
November 4, 2024
Stand-in for a blog, will replace soon.