I've created Diagify, a CLI for converting natural language to technical diagrams. There are a bunch of diagram-as-code tools out there and I had the idea that it should be possible to use LLMs to generate the diagram code based on a natural language description, execute it, and return the generated image. This is essentially what Diagify does.

More specifically, Diagify generates code for the Mingrammer library which is specifically for technical diagrams. It also uses the OpenAI API to generate the Mingrammer code from the natural language description.

The workflow is first OpenAI generates the Mingrammer python code based on the description, then it does some basic error checking. It was found that often the generated Mingrammer code had incorrect imports so this is checked. If incorrect imports are found, OpenAI is called again for correction with the incorrect imports identified and also some suggested imports to replace them with. Then the Mingrammer code is executed. If it is executed successfully the corresponding image is generated. If there is a runtime error then OpenAI is called again with the identified error in an effort to correct it. It's become somewhat reliable at this point.

My reasons for creating Diagify are two-fold 1) to see if it even would work and 2) creating technical diagrams by hand or even using diagram-as-code tools can be tedious so using a simple natural language interface could be helpful.