Here are the most annoying things about this:

1. It consumes a ton of unnecessary output tokens 2. It is limited to 4k output tokens 3. Sometimes modifies parts that I did not tell it to 4. IT IS SLOOOW (imagine we wanted to change just a word at the beginning)

What if, we could just patch our changes using a diff? I mean this is nothing groundbreaking, but still it seems, as if no one actually built something reusable or open.

So here it is: LLM Patcher (kinda dumb name I know)

Here is how it works

1. The user provides a text and a find-and-replace query. 2. The text is split into lines and sentences. 3. Each line and sentence is then prefixed with an identifier that looks like <l1s1> for line 1, sentence 1. 4. The LLM is then asked to find-and-replace the query in each line and sentence. 5. The changes are then streamed back to the user in the form of a diff. The diff looks like <r:l1s1> string to find || string to replace.

Let me know what you think about it and please give it a star

Cheers, Luke