Show HN: WarpParse – Rust ETL engine 1.5–8x faster than Vector

2 points

4 months ago

Hey HN, We’ve been working on WarpParse, an open-source (Apache 2.0) ETL engine built in Rust, and we’re excited to share it with you today. We built it because we found that existing tools often struggled with resource efficiency or configuration complexity when handling massive log volumes. WarpParse aims to solve this with: Performance: 1.5x to 8x higher throughput than Vector in our benchmarks. Efficiency: Uses ~1/3 the CPU and ~37% of the memory of Vector under similar loads. DSLs: We created WPL (Warp Parse Language) for strong-typed parsing and OML (Object Modeling Language) for declarative data enrichment with native SQL integration. We’ve also built a visual editor (WpEditor) to make rule-writing less of a headache. We’d love to hear your thoughts on the architecture, the DSL approach, or any feedback you have after checking out the repo. GitHub: https://github.com/wp-labs Editor: https://editor.warpparse.ai As a fledgling open-source project, we deeply recognize the power of the community. We hope WarpParse proves helpful to you!Thanks!