Small Models Struggle with Tool Calling
Smaller language models are fast, lightweight, and cost-effective, but they often stumble when it comes to reliable tool calling. Unlike larger models, they frequently output malformed arguments, miss required parameters, or fail entirely when an API throws an error.
Fortunately, you can bridge this reliability gap. Here are two practical ways to help small models execute tool calls correctly.
1. Make Tool Errors Verbose and Descriptive
When a tool call fails, small models often panic or loop because the error message is cryptic (e.g., `Error 400: Bad Request`).
To help the model self-correct, wrap your tool execution layer in descriptive, human-readable error messages. Instead of returning a raw stack trace, feed back a structured explanation of *why* it failed. For example:
* **Bad Error:** `KeyError: 'code'`
* **Helpful Error:** `Missing required parameter 'code'. The browser_exec tool requires a 'code' argument containing Python code to execute.`
Giving the model clear feedback allows it to adjust its parameters on the next turn rather than repeating the same mistake.
2. Teach the Model with Concrete Few-Shot Examples
Small models rely heavily on pattern matching. If you simply give them a tool schema and hope for the best, they will likely guess the syntax incorrectly.
Use prompt engineering to show them the exact formatting, syntax, and structure you expect. Provide clean, copy-pasteable examples directly in your system prompt.
Example Prompt Template
Use `browser_exec` for browser automation.
The tool accepts:
- `code`: Python code (required)
- `session`: Browser session name (optional)
- `timeout_s`: Timeout in seconds (optional)
Example:
browser_exec("print(js('document.body.innerText'))")
browser_exec("print(page_info())")
Comments