November 17, 2024
ZeRO to Hero: Building FSDP From Scratch
This blog will walk through the motions of rebuilding the ZeRO optimizer and FSDP from scratch.
I'm an ML Engineer based in Boston, Massachusetts. I specialize in training large language models.
I work on AI systems in the NLP space. Some of my past projects have included large-scale LLM training, RAG-based tools, backport systems, and more.
I develop full-stack React/Next.js appllications that transform complex solutions into accerssible, sharable tools.
Take a look below at some of my featured work for clients from the past few years.
Here are some of my recent blog posts: