My name is Salman Paracha. I aam the the Founder/CEO of Katanemo - the organization behind the open source Arch GW (an intelligent gateway for prompts - https://github.com/katanemo/arch). Today, we are making the (SOTA) LLMs engineered in Arch GW for function calling scenarios available under an OSS license that borrows from Llama's community license.

What is function calling? Function calling helps developers personalize apps by calling application-specific operations via user prompts. This involves any predefined functions or APIs you want to expose to perform tasks, gather information, or manipulate data - via prompts. With function calling, you get to support agentic workflows tailored to domain-specific use cases - from updating insurance claims to creating ad campaigns. Arch-Function analyzes prompts, extracts critical information from prompts, engages in lightweight conversations with the user to gather any missing parameters and makes API calls so that you can focus on writing business logic.

Arch-Function is an auto-regressive model that if run on the NVIDIA A100 GPUs using vLLM offers throughput of ~1900/output tokens per second, and a output token price of $0.10/M token. This is ~12x faster and 44x cheaper than GPT-4o.