Fine-tuning Llama 2 to code like me

4 points

3 years ago

Hey HN, I started live streaming myself coding a couple weeks ago and began a series where I’m trying to train an LLM to code like me.

What I mean by coding like me is I build a dataset off of my GitHub data. Using PEFT I’m able to fine tune one of these smaller scope LLMs on Colab.

Here’s my most recent stream https://m.youtube.com/watch?v=TYgtG2Th6fI&t=3998s

And my initial stream a few days ago where I show how I built my dataset and structured the fine tuning script https://m.youtube.com/watch?v=ozOsrX9d0a0&t=3747s

If you want to hang out and code with me tune into my live stream and let me know how ugly and NOT 10x my coding is!