I recently developed an open source memoization library for JavaScript. I've extensively tested it and compared it against other popular memoization packages and in my benchmarks it outperforms existing packages.

What really surprised me was how the existing packages on npm had benchmarked their own performance - and some of them show impressive results for their package which did not have impressive results in my own benchmarking.

What I found was that many benchmark their memoization by feeding the memoized function with one and the same argument millions of times. To me that is not a realistic use case - you would just put the result of the function call in a variable.

My benchmarking strategy is to create a set of 1000 different arguments and have the memoized function return results for those arguments. So the first 1000 function calls would call the original expensive function - and all consecutive calls would hit the cache.

If you benchmark like this you get totally different results and some libraries that are very fast when the memoized function is called with one and the same argument get pretty slow to a point where it might be better to call the original expensive function.

How do you think should memoization be benchmarked and what are your use cases for memoization of pure functions?

You can find the benchmarking results on npm: https://www.npmjs.com/package/sonic-memoize

You can find the benchmarking code here: https://github.com/Animus-Blue/sonic-memoize/blob/main/benchmark/index.ts

Thanks for your time!