Show HN: I reproduced Code Llama fill-in-the-middle code completion training

2 points

2 years ago

As part of my master's thesis on optimizing LLMs for code completion in OpenAPI format, I implemented a script for fine-tuning Code Llama for Copilot-style code completion. Surprisingly, I was not able to find a similar implementation before since most works focus on instruction fine-tuning for chat-like interaction.

Related publication: https://arxiv.org/abs/2405.15729 Code Llama fill-in-the-middle fine-tuning: https://github.com/BohdanPetryshyn/code-llama-fim-fine-tunin... OpenAPI completion benchmark: https://github.com/BohdanPetryshyn/openapi-completion-benchm...