* ChatGPT web requires a lot of copy-paste, it rewrites whole document even if you need to update a part of it, etc. * Github Copilot completions are rather unreliable and do not leave an option to specify what you want; Copilot's chat sidebar is little more than ChatGPT integrated into the IDE * Google Docs have right UI for non-code text, but they use really dumb model (not Gemini 1.5 Pro). Also won't work for code.

So... I wrote a Emacs Lisp function which calls LLM with contents of the buffer to generate text according to the (optional) user's command. It can both insert new text and update selected text.

In my twitter post I have two videos demonstrating the usage: https://x.com/killerstorm/status/1791840684208463940

I haven't been using Emacs for 10+ years (IntelliJ offers better ergonomics for Kotlin and TS), but it does seem like the easiest platform for this kind of stuff.

Took me about 4 hours to implement this, largely because I knew almost nothing about Emacs Lisp, my Common Lisp is rusty (I've used it professionally 10+ years ago) and I decided to use CL server as a proxy for the LLM API. It's likely unnecessary, and it might be possible to achieve the same in pure Emacs Lisp. But I wanted to use more advanced prompting down the line (e.g. feed it adjacent files, chained prompts, etc) and for me it would be a lot easier to do it in CL.

Regarding LLM models, OpenRouter offers quite a wide selection and they are easy to switch, so I tried >10 of them. I've got best results with google/gemini-flash-1.5 and openai/gpt-4o. Video demo uses Gemini. GPT-4o seems to be just as fast, and it handles formatting in a cleaner way (i.e. it keeps track of whitespace), also probably a bit smarter. Clause 3 Opus is a good model but it's slow.

Llama 3 70B Instruct is fast but you lose some intelligence.

Most other models, including 8B Llama 3 Instruct did not understand the prompt... It can be simplified, sure, but do I really want to use a model which cannot understand "You should return the text to be insert in between `before` and `after`?" I tried simpler prompts (i.e. without JSON), and they just struggle with understanding what they need to return. Pretty funny that people said that "mistralai/mixtral-8x7b-instruct" is "almost GPT-4", but it cannot even...

Anyway, here's code which you might try at home: https://gist.github.com/killerstorm/0fddba22d778c871452213b1...

CL server can trivially be replaced with Python.