Show HN: Open-source load balancer for distributed MCP server architecture

2 points

a year ago

Like many others, I’ve been hacking on MCP servers lately. The issue I ran into was that running multiple MCP servers behind a unified backend was hard. I needed a way to combine all the tool calls from different MCP servers so the client got a unified view, route requests to the correct server based on the tool call, and maintain session affinity for SSE requests. This led me to build CATIE, which solves the above issues.

CATIE is a lightweight proxy that routes MCP requests to the appropriate backend services based on their content.

Key Features

- Content-Based Routing: Routes requests based on tool call. This allows server operators to use a micro-service architecture, with the user only installing one server. This separation allows server operators to scale tool calls independently.

- Unified Tool Call Response: Combine the tool list response from multiple MCP servers. - Session Stickiness: Maintains client connections to the same backend

- Pattern Matching: Uses regex patterns to route tool requests

- Real-time Monitoring: Simple dashboard to see traffic patterns and performance with built-in Prometheus integration.

- Backend Switching: Change where requests go without client reconfiguration

How It Works

CATIE sits between your clients and your MCP servers. When a request comes in, it:

- Parses the JSON-RPC request to understand what it's trying to do

- Applies your routing rules to determine the appropriate backend

- Forwards the request to the backend

- Maintains session stickiness for ongoing conversations

- Has a built-in UI for monitoring statistics and integrates with Prometheus

CATIE is fully open source under the MIT license. Contributions, feedback, and feature requests are all welcome!

- GitHub: https://github.com/mclenhard/catie-mcp - Docs: https://www.catiemcp.com/docs/