Last year around this time I was joking with people if you can hack together something like SparkSQL with all the best Python open-source: Polars/Arrow/Ray/SQLGlot.
So here it is.
Definitely NOT production ready -- but should work for most advertised use cases.
Only has a dataframe/datastream API right now, working on full SQL support. Hard because of all the weird SQL people write :-P
1 comment
Show HN: Rewriting SparkSQL in Python | Heykuki News