He starts with the basics, implementing a neural network and backpropagation from scratch and continues with the basics of language models and before you know it you will more or less have implemented GPT-3, somewhat simplified.

I've been following the course over the last few days and I'm finding it really great. To follow along you should watch the videos on one screen and have another screen or your laptop setup with jupyter to type in the code line by line. I find it very helpful to take lots of pauses, explore the code that's been written a bit and try to guess what's coming next.

All the videos are at this YouTube playlist https://www.youtube.com/playlist?list=PLAqhIrjkxbuWI23v9cThsA9GvCAUhRvKZ