1. 01A GPT in C, end to endPart 1: a dependency-free character-level GPT in C that trains on a text file and generates names — made reproducible before it was made fast.
  2. 02The first vertical slice: a terminal chatPart 2: building the whole chat loop on the CPU before any GPU work, so the destination was defined before the road was built.
  3. 03A device boundary, before any devicePart 3: moving every buffer and operation behind an interface while the CPU still did all the math, so CUDA could land later without touching the model.
  4. 04First CUDA kernels: correct before fastPart 4: bringing up one GPU inference operation at a time, matching the CPU to 1e-4, and discovering the first GPU build was 4.9x slower.
  5. 05The GPU is 4.5x slower, and the profile says whyPart 5: putting the chat loop on CUDA, measuring it honestly, and finding that 84% of the API time was spent shuttling activations across the bus.
  6. 06Training on the GPU: 762,936 launches per stepPart 6: moving backward propagation and Adam onto the device, matching the CPU to 4e-05, and finding the step was launch-bound rather than compute-bound.
  7. 07Batching the window: 547 ms to 2 msPart 7: turning a training window into matrix problems across all positions, cutting a step by about 270x with an identical loss curve.
  8. 08Subwords beat characters, and the tokenizer is a quality leverPart 8: byte-level BPE, then GPT-2's real vocabulary and SentencePiece, and the runs that showed compression moves quality more than architecture does.
  9. 09Rotary embeddings: an honest no-winPart 9: replacing the learned position table with rotary encoding, proving it with a mathematical oracle, and shipping it despite no loss improvement at the small size.
  10. 10Fusing to parity, and hitting one kernelPart 10: a device-resident workspace and three fused operations took inference from 4.0 ms to 2.52 ms per token, exactly level with the CPU.
  11. 11Seeing inside the GPU while it runsPart 11: live device metrics, Prometheus and Grafana, NVTX ranges and an in-process CUPTI subscriber, all without changing the default build.
  12. 12Feeding it two gigabytesPart 12: making the runtime load the full 2.08 GiB TinyStories corpus, and learning that data scale beats model scale at this size.
  13. 13Long context without retraining, and the null resultPart 13: serving a 512-token model at 4,096 positions with four angle-scaling modes, and finding that every published trick made it worse.
  14. 14Teaching it to call tools, and the form it fills wronglyPart 14: a chatml-tools template, a JSON logit mask with a forced opener and a bounded agent loop, plus the held-out table that says the mechanism works and the model does not.
  15. 15Real prose, and what is left to buildPart 15: measuring the same model on text people actually read, then the honest map of what this series did not reach.