Harbor is a toolkit to run LLMs and related services locally. It's not designed as a deployment solution, rather as a convenient local environment.

I was rocking a very similar setup (less components and features though) and then decided to polish it a little bit and make it open source.

Few interesting details about the project:

- Dynamic sub-selection of compose files to run together depending on which services are currently running. For example, Harbor knows that you're running SearXNG and Open WebUI together and injects a config for Web RAG based on SearXNG into the WebUI, same for TTS and LLM backends

- CLI supports arguments scrambling via a retry, e.g. `harbor set config` and `harbor config set` run the same command, it's supported for nearly all first pairs of arguments to the CLI

- Being an LLM project - after the setup you can ask CLI how to do something: `harbor how to configure vllm model?` (yes, no quotes), 50% gimmick, but very fun

Cheers, Ivan

P.S. Forgive me for all the shell sins in the source.