Show HN: A tiny reasoning layer that steadies LLM outputs (MIT; +22.4% accuracy)

6 points

10 months ago

We kept shipping “simple” LLM features that were fluent-but-wrong. After too many postmortems we wrote down the failure patterns and added a small reasoning layer in front of the model. It’s model-agnostic, sits beside your existing stack, and you can implement it from a single PDF (MIT).

What’s inside the PDF

A problem map of 16 failure modes we kept hitting in real systems (OCR/layout drift, table-to-question mismatches, embedding≠meaning, pre-deploy collapse, etc.).

Four lightweight gates you can add today:

Knowledge-boundary canaries (empty/adversarial/known-fact probes).

ΔS “semantic jump” check to catch fluent nonsense when the draft answer drifts from retrieved context.

Layout-aware anchoring so chunking across PDFs/tables doesn’t silently break routing.

A minimal semantic trace for incident review (tiny, not full transcripts).

Bench snapshot (same model, with vs. without gates): Semantic Accuracy ↑ 22.4% · Reasoning Success Rate ↑ 42.1% · Stability ↑ 3.6×.

Traction (last ~50 days)

~2,400 downloads of the PDF.

~300 cold GitHub stars on related material (no marketing burst).

Also received a star from the creator of tesseract.js, which was nice validation from the OCR world.

Why this might be useful to you

You don’t need to swap models or vendors. The PDF describes checks you can drop into any RAG/agent/service pipeline.

No servers, SDKs, or proxy layers—just logic you can copy.

Link is Git Repo

Happy to answer HN-style questions (what breaks, where it fails, ablations, how we compute ΔS, etc.). If you try it and it doesn’t help, I’m also interested in the counter-examples.

with Terrseract (OCR legend) starred it verify it, we are WFFY on top1 https://github.com/bijection?tab=stars