I wrote other scripts, compiled personal eval sets, and subscribed (then unsubscribed) to a bunch of online tools. Nothing seemed right.

Until last Friday when, talking about side project ideas with my girlfriend, she said, “What if it was in your code editor?”

And I thought, wow, I love this woman.

This VSCode extension instead lets you highlight your prompt text and run it against any and all models [2]. You can toss in your API keys, which are only stored locally, and use it for free.

Any feedback would be invaluable - this is my first extension!

I’ll be adding the same highlighting functionality I have for prompts to schema validation, where you can highlight any form of schema (pydantic, zod, json), and I’ll use a small llm to parse it + format it for different providers.

[1] https://github.com/benguz/prompt-octopus [2] https://promptoctopus.com