The idea is that you can evaluate how new policies will perform by using historical data generated under a previous policy. For example, rather than testing a new fraud policy in an A/B test, you can use historical logs to determine if the new policy will outperform the existing one. This seems like it could be a great step before A/B testing new policies.

I whipped up some example code to test out what would be considered the "hello world" of offline policy evaluation if anyone is curious: https://github.com/banditml/offline-policy-evaluation/blob/main/examples/Inverse%20propensity%20scoring.ipynb

My question to you is -- have any of you have tried this or do any of your currently use OPE at your companies?