Finetune language models 30x faster

2 points

3 years ago

Hi HN! Today we're launching our AI startup focusing on creating cool AI products! Our first launch is Unsloth which allows you to train LLMs 30x faster, use 60% less memory with 0% loss in accuracy and requires NO hardware changes!! Train SlimOrca in 54 hours instead of 54 days!

We hand derived backpropagation steps, did some smart chained matrix multiplication bracketing, wrote all kernels in OpenAI’s Triton language, and applied lots of maths and coding trickery!

We have an open source version which finetunes Llama 2x faster and uses 50% less memory. Have a try at https://github.com/unslothai/unsloth. Any feedback would be appreciated! Discord: https://discord.gg/nsS4V5Z6ge