Show HN: We're building Apache spark for agents with Rust and Datafusion

2 points

a month ago

Hi HN, BT here. We are trying to build a data platform that's dedicated to serve agents, so we call it the apache spark for agents. The reason why we think we need a new data platform for agents is that we think existing platform is restricting how agents are utilizing data,

and we think we should provide agents full autonomy on how to use data and what data to use, and it should have a better leverage for agent's ability.

We chose datafusion because it's great extensibility and performance, so our data platform can connect with various different data sources.

If you find this project interesting and want to give a try, you can start with the auto knowledge base skill: https://github.com/SkardiLabs/skardi-skills/tree/main/auto_k... which can help you turn your documents into a instant knowledge base that local agents like openClaw or Hermes can ingest instantly.

Anyways, happy to answer any questions.

1 comment