Show HN: Goxe – Fast log clustering on an i5 (Reduced to 1 alloc/log, road to 0)

2 points

3 months ago

I’ve just released Goxe v1.4.0. After the last update, I’ve been obsessed with squeezing every bit of performance out of Go.

The Big Update: Offline Normalization (-brew) I added a new mode to process and normalize legacy logs that were sitting on disk before Goxe was installed. It clusters similar messages, reduces storage footprint, and ships metrics to a remote server.

The Engineering Win:

    Allocations: I managed to reduce the overhead from 2 allocs/op to just 1 per log line using unsafe zero-copy string conversions and bufio.Scanner optimization.

    The Goal: I’m currently refactoring the core pipeline to hit 0 allocations in the next cycle.

    Performance: Still hitting 19k logs/s on an old i5-8250U @ 3.40 GHz with a minimal RAM footprint.

Why use it? If you have massive log files cluttering your disk, -brew will normalize them into a structured summary ([name]_[date]_normalized.log), saving space and giving you clear stats (Count, First/Last seen) without killing your CPU.

I’d love to hear your thoughts on the zero-copy approach.

Repo: https://github.com/DumbNoxx/goxe