I, like many of you, have really struggled with benchmarking and comparing models, from missing code, to opaque experiment parameters slowing the process. I decided to take matters into my own hands, and created a mini-toolbox in my free time to streamline the process.

The goal is to reduce the time and effort spent on replicating experiments, enabling researchers to focus on innovation rather than setup.

Knowing this task is not an easy feat, I reached out to the community a while ago and got tremendous help already. With the team, we now have added a code extractor, an AI experiment description generator, and a custom script editor to the functionalities we already implemented.

The project is open-sourced, and you're so welcome to share your thoughts with us, contribute, or leave a star if you like what we're building :)

You can find the repository here : https://github.com/ReproModel/repromodel

Thank you for your time, and feel free to leave any comments or suggestions below!