Show HN: Agents.json – Open-source API specification for LLMs

4 points

a year ago

Hey HN, we’re building an open specification (https://github.com/wild-card-ai/agents-json) that lets agents discover and invoke APIs with natural language, built on the OpenAPI standard.

Here’s a walkthrough of how it works: (https://youtu.be/kby2Wdt2Dtk?si=59xGCDy48Zzwr7ND).

There’s 2 parts to this:

1. The agents.json specification describes how to link API calls together into outcome-based tools for LLMs. This file sits alongside an OpenAPI file.

2. The runner package loads agents.json files as tools for an LLM that can then be executed as a series of API calls.

We’ve put together some real examples if you're curious what the final output looks like. Under the hood, these LLMs have the same system prompt and we plug in a different agents.json to give access to different APIs. It’s all templatized.

- Resend (https://demo.wild-card.ai/resend)

- Google Sheets (https://demo.wild-card.ai/googlesheets)

- Slack (https://demo.wild-card.ai/slack)

- Stripe (https://demo.wild-card.ai/stripe)

We really wanted to solve real production use cases, and knew this couldn’t just be a proxy. Our approach allows devs to make API calls from their own infrastructure. The open-source specification + runner package make this paradigm possible. Agents.json is truly stateless; the client manages all memory/state and it can be deployed on existing infra like serverless environments.

You might be wondering - isn’t OpenAPI enough? Why can’t I just put that in context?

We thought so too when building an agent with access to Gmail. Putting the API spec into context gave us poor accuracy in tool selection and in tool calling. Even with cutting down our output space to 5-10 endpoints, we’d see the LLMs fail to select the right tool. We wanted the LLM to just work given an outcome rather than having it reason each time which series of API calls to make.

The Gmail API, for example, has endpoints to search for threads, list the emails in a thread, and reply with an email given base64 RFC 822 content. All that has to happen in order with the right arguments for our agent to reply to a thread. We found that APIs are designed for developers, not for LLMs.

So we implemented agents.json. It started off as a config file we were using internally that we slowly started adding features to like auth registration, tool search, and multiple API sources. 3 weeks ago, Dharmesh(CTO of Hubspot) posted about the concept of a specification that could translate APIs for LLMs. It sounded a lot like what we already had working internally and we decided to make it open source. We built agents.json for us and we’re excited to share it with you.

In the weeks since we’ve put it out there, agents.json has 10 vetted API integrations (some of them official) and more are being added every day. We recently made the tool search and custom collection platform free for everyone so it’s even easier for devs to scale the number of tools. (https://wild-card.ai)

This specification isn’t perfect and can’t be without your feedback. Please tell us what you think! Especially if you’re building agents or creating APIs.

Many thanks, Yagnya and Kaushik from Wildcard