Crawls sitemaps, parses metadata files (robots.txt, humans.txt, llms.txt), detects tech stack with Wappalyzer, then generates summaries using either AWS Bedrock or a local Llama model via Ollama.

Batch mode processes CSVs with checkpoint/resume - hit Ctrl+C anytime and pick up where you left off.

The local LLM option means zero API costs and your data never leaves your machine. Llama 3.2 3B works surprisingly well for this task.