It's the #1 most requested thing I've been hearing from open source model builders: the ability to use multiple GPUs with QLoRA training. So that's why we decided to make it our first project.

Huge thanks to Tim Dettmers for helping us get started to this -- and of course for creating QLoRA in the first place!

Let me know if you have any questions or thoughts.