To get started (if you've done your code scan due diligence and you're setup on CUDA), you can just run `git clone https://github.com/tysam-code/hlb-CIFAR10 && cd hlb-CIFAR10 && python -m pip install -r requirements.txt && python main.py` and you should see things training straightaway. If you have a good internet connection and card, your first run should be done within 90 seconds (including dependency installs, dataset download, etc).

Project Introduction

Hi there,

I've been working in the modern instantiation of this field for just over half a decade or so, and have always sort of wanted a good testbench to develop neural network research on. Most of the codebases I've worked with over the years have been very ad-hoc or not very close to the bleeding edge. Or, the highly performant codebases really weren't amenable to hacking -- just for performing the main function they were built for (a single paper, benchmark, service, etc).

I've spent 120-130 hours or so rebuilding David Page's lovely record-setting dawnbench project (https://myrtle.ai/learn/how-to-train-your-resnet/) in a way that is generic and amenable to both hacking and rapid experimentation/prototyping. One benefit of this is that the original code is extremely performance optimized, so not only is it an easy workbench to prototype on, but a good testbed for if a new method is orthogonal or not to older methods.

I've tried to make it as absolutely plug and play as possible, and I hope this offers a nice combination of being fun and useful. I've already gotten a significant amount of value in testing some methods that would otherwise be very hard to test with it. In both cases, either change required just a single line to be added, which I felt validated the original design decisions. I have some hopefully-not-to-nefarious plans for it if I can get it down to the ~2 seconds mark within a few years.

I have a Patreon for those interested in seeing more similar work from me in the future. You can get that and other info from the github link above.

I plan on hanging around in the comments for a while in case anyone has any questions. Please do let me know, I love the conversation, and whether or not you like/liked it, or if you find this intriguing or not -- this helps me build better software in the future! So please, feel free to drop a line. I'd love to hear what you think! :) :D