Show HN: Autopen, token-tree text editor that runs LLMs and shows probabilities

4 points

a year ago

This is a program I have been working on that lets you interact with an LLM (in llama.cpp's GGUF format) locally using a text editor buffer. It tokenises the buffer on the fly, visualises the model's conditional probabilities for every token, and lets you view and jump back and forth between generated or manually entered continuations in descending order of probability at every point.

Compared to some other tools that were posted here recently, it operates at a rather lower level - the branching points of the tree are single tokens=words/word fragments rather than "prompts" and "responses", and you can see the probability/surprisal levels for every single token of a text that you can edit, which opens up some interesting applications.

I have made a video showcasing its use to investigate and debug a small DeepSeek-R1 distill's reasoning process here: https://www.youtube.com/watch?v=GXWZPpVI0zU . Towards the end of the video, I also briefly address a more out-there application (natural-language reasoning "spreadsheets").

I'm making the source available under GPLv3, and there are Windows binary builds available (depending on your CUDA situation, installing them might not be a one-click affair as you may need to grab the appropriate version of llama.cpp DLLs for your system).