Phi 3.5 was just released, and is distilled from GPT4. Unsloth makes finetuning 2x faster, uses 70% less VRAM + has no accuracy degradations. We rewrite all backprop steps and reduce FLOPs and write everything in Triton (JIT low level CUDA).

If you want to own the weights after fine-tuning, give Unsloth a spin! I have free Colabs and Kaggle notebooks as well at https://github.com/unslothai/unsloth