After having it lingering in a private repo for a while, I decided to open source Snapdir, a reasonably documented and tested set of bash scripts for creating, sharing, and verifying snapshots of directories and their contents using human-readable manifests.

While I created Snapdir to solve a specific problem around ML pipeline reproducibility and data distribution, I've also found it very useful for vendoring dependencies such as VM AMIs, binaries, and container images in my infrastructure-as-code environments.

The manifest format should be easy to understand and kept under version control for auditing purposes. If you want to incorporate it into your workflows, you can use the snapdir-manifest script in isolation.

The project is still in its early stages, and the current bash version should be considered a proof of concept. You can use the tests included in the bash version to validate implementations in other languages.

For a quick test, you can play with it from the 5MB docker image:

    docker run -it --rm bermi/snapdir help

    docker run -it --rm bermi/snapdir manifest /lib