It checks for signs of hallucinated or unreliable output using a multi-method approach (overconfidence patterns, factual density, coherence, contradictions, etc).

What it does:

Works with GPT, Claude, local models (e.g., Mistral, DialoGPT)

Outputs a hallucination probability (0.0–1.0)

Flags overconfident or uncertain language

Scores factual density, coherence, and contradictions

Compares responses to context (if provided)

Fully framework-agnostic — no extra dependencies

Built for production + research workflows

Benchmarked on 1,000+ samples:

F1: 0.75

AUC-ROC: 0.81

Fast: ~0.2s per response

Comes with plug-and-play examples:

OpenAI, Anthropic, local models

Flask API

Custom scoring configs

I’m giving this away free under MIT. Would love feedback, issues, PRs — or just to know if it helps you build safer LLM apps.

GitHub: https://github.com/Mattbusel/LLM-Hallucination-Detection-Scr...