Most Raft/Paxos libraries treat the state machine as a black box. That’s fine until the state machine does real-world things like charging a card, sending an email or firing a webhook.

Then the leader crashes after the side effect but before commit, and you get duplicates. Every time. All the “exactly once” hand-waving collapses the moment a process dies mid-flight.

This repo is my attempt to fix that by design: side effects live in the replicated log, not behind it.

how it works, roughly:

Side effects are written as pending entries in replicated state. Only the current leader executes them, under a fencing token.

Leadership is fenced durably so a zombie leader can’t wake up and re-run old effects.

The state machine runs with a deterministic RNG seed + block time from the log, so replay is bit-for-bit or it halts.

The WAL is strict: CRC + hash-chained. Corruption stops the system instead of guessing.

Trade-offs are explicit:

Effects are at-least-once. Exactly-once requires stable IDs and sink-side dedup.

CP over AP. Safety over availability.

If you’ve ever been paged because a leader died at exactly the wrong microsecond, you know why I built this.

Repo: https://github.com/abokhalill/chr2

Happy to answer any questions about chr2 architectural internals or design trade offs.