Ask HN: Open-source equivalent of Millwheel

2 points

11 years ago

I've been reading the Millwheel [1] paper, and am not aware of any similar, open-source technology that provide feature parity. Specifically, exactly-once delivery and strong productions seem like very nice capabilities (i.e. they're hard to DIY correctly).

I search HN for previous discussions of Millwheel, and have read through some of the intro documentation for Samza [2], AWS Kinesis [3], and Apache Storm [4], but it's not immediately obvious which features each supports.

This is also related to my submission the other day on making Storm idempotent [5].

Is there a thorough comparison somewhere, or a feature matrix (c.f. these sorts of eye-bleeding tables: [6])?

[1]: http://research.google.com/pubs/pub41378.html [2]: http://samza.incubator.apache.org/ [3]: http://aws.amazon.com/kinesis/ [4]: https://storm.apache.org/

[5]: https://news.ycombinator.com/item?id=8723478 [6]: http://en.wikipedia.org/wiki/List_of_AMD_accelerated_processing_unit_microprocessors#Desktop_processors