When analyzing data in a Jupyter notebook, I unconsciously memorize "rules of thumb" to determine if my results are correct. For example, I might print some summary statistics and become skeptical of some outputs if they deviate too much from what I've seen historically. For more complex analysis, I often create diagnostic plots (e.g., a histogram) and check them whenever new data arrives.

Since I constantly repeat the same process, I figured I'd code a small library to streamline this process. nbsnapshot benchmarks cell's outputs with historical results and raises an error if the output deviates from an expected range (by default, 3 standard deviations from the mean).

To learn more, check out the blog post (https://ploomber.io/blog/snapshot-testing/).

I quickly put together this as a proof of concept and would love to learn if this sounds useful for other notebook users so I spend more time working on it. Please share your feedback!