The Smol Training Playbook by Hugging Face explains what it’s really like to build powerful AI language models, like ChatGPT or Llama. It focuses on SmolLM3, a 3-billion-parameter model trained on 11 trillion words, and shows that training AI isn’t just fancy math—it’s full of trial and error, crashes, and teamwork.
Read the details https://huggingface.co/spaces/HuggingFaceTB/smol-training-playbook
The authors start with a simple question: should you even train your own model? Today, many excellent open-source AIs (like Qwen, Gemma, and Llama) already exist. Instead of copying them, you should only train a new one if you’re doing new research, need a model for a very specific job (like medical or legal text), or must control all the data for safety or privacy reasons.
Once you know why, the next step is what to train—deciding the model’s size, type, and data. Then comes how—setting up computers (GPUs), choosing frameworks like Megatron or DeepSpeed, and running small tests (called ablations) to see what works. The team behind SmolLM3 used a careful, step-by-step approach: testing one change at a time, tracking results, and focusing more on high-quality data than on fancy architecture tricks.
The main message: building an AI is like training an athlete. You need a goal, a plan, and lots of practice. The best teams move fast, learn from mistakes, and keep improving. The playbook turns complicated AI training into a clear, honest guide for anyone who dreams of building their own world-class model.