Ask HN: How to Improve LLM Function Calling?

2 points

a year ago

If we want to give an LLM access to external tools, it seems like the current best method is to use some form of function calling (tool-use in OpenAi’s case). This essentially provides the llm a context of a list of functions it can call and is prompted to return a structured output if it needs to call a function in the list. The backend code then catches these cases and renders the code and feeds in back to the llm. This is great for validating that LLMs can understand how to use different tools, but it seems to me that this setup is quite limited in actual use cases for several reasons:

1. It requires the developer to curate descriptions for the function (example below), what happens when you want to give it an entire private repository of functions?

Example of a function in OpenAi’s Tool-Use (https://platform.openai.com/docs/guides/function-calling): { "name": "get_delivery_date", "description": "Get the delivery date for a customer's order. Call this whenever you need to know the delivery date, for example when a customer asks 'Where is my package'", "parameters": { "type": "object", "properties": { "order_id": { "type": "string", "description": "The customer's order ID." } }, "required": ["order_id"], "additionalProperties": false } }

2. What if your action requires a more sophisticated series of functions nested with each other? Currently LLMs output a single function call.

3. More complex use-cases require fine-tuning models to more accurately detect which functions to use/not use. Doing so requires some form of generation of synthetic data, which grows rapidly with the number of functions you give it access to.

I'm wondering if anyone has worked on better methods for function calling, specifically giving an LLM access to a library of functions without having to input each function+description? Feeding them as context also wouldn't work here given limitations in context windows, and RAG isn't able to understand the functions deeply enough.