Show HN: Kanoniv – Identity resolution in 170 lines of YAML Built in Rust

1 point

4 months ago

I kept rebuilding the same identity resolution pipeline at every company i have been at. Normalize emails, block on name+phone, score pairs, cluster, survivorship. 350+ lines of SQL each time, and it still missed half the matches.

Kanoniv is a declarative identity resolution engine. You write a YAML spec (sources, blocking keys, scoring weights, survivorship rules) and it runs a Fellegi-Sunter probabilistic matcher in Rust via PyO3.

I published a comparison repo with the same 6,500 records resolved 3 ways: dbt SQL, Splink, and Kanoniv, so you can see the tradeoffs yourself: https://github.com/kanoniv/kanoniv/tree/main/examples/custom...

Free local SDK (pip install kanoniv).

Happy to answer questions about the matching algorithm or the Rust/PyO3 architecture.

Moving the heavy lifting to Rust via PyO3 is a smart move for performance especially when dealing with the O(n2) nature of record linkage.