Show HN: PromptProof – CI gate for LLM outputs (schema/regex/cost; no API keys)

news.ycombinator.com

2 points

9 months ago

I built a tiny GitHub Action that fails PRs when LLM output breaks contracts. No live model calls in CI... it runs on recorded fixtures. What it does:

Deterministic checks: JSON schema, regex, list/set equality, numeric bounds, file diff

Snapshots + regression compare

Cost budget gate

PR comment + HTML report

Try it in a minute: copy the Quick Start, open a PR, see red → fix → green.

Links: Marketplace: https://github.com/marketplace/actions/promptproof-eval Demo repo: https://github.com/geminimir/promptproof-demo-project Sample report: https://geminimir.github.io/promptproof-action/reports/befor...

Looking for blunt feedback... Was onboarding smooth? Anything missing you expected? Is the report clear enough to make this a required check?