I’ve been working on a lightweight log collector which will run on any vps, is extremely easy to set up, and works with your current libraries.

Most log platforms either

- are expensive (datadog, newrelic) - require you to use opentelemetry, which is difficult to set up and not well documented - require you to host a clickhouse instance

For someone who has projects with less than 5,000 users, I failed to find a simple log platform which worked with the log libraries I already used.

Erlog is a simple python script which accepts logs from an HTTP endpoint, and optionally watches log files and inserts JSON formatted logs into a local duckdb database. You can then query logs using natural syntax.

There is also support for `parent_id` fields in logs, so you can track execution across many different services (see github).

This is MIT licensed. I’d be interested in any feedback you have, and any ways I can improve erlog. I’m thinking of adding support for metrics in the future.

The GitHub link is: https://github.com/Nevin1901/erlog