Does a local model refuse tool calls, or does it invent ones that do not exist?

It invents them. A local model is more likely to emit a tool name that is not on the list than to refuse. The name will look plausible: a synonym, a tool it saw in some other product’s docs, or a second step it wishes you had provided. Refusal is the failure you want, because nothing runs. Invention is the failure you get. It is only safe if the runtime rejects unknown names and does not map them onto a shell or a browser. If the runtime turns the text into a chat reply, you can think it refused when it tried.

I do not trust a local agent until I have seen an unknown name die in the trace, not in the model’s apology.

The tool it wished I had

The list was three names: list_dir, read_file, edit_file. The task was to rename a key in one config file. The model’s first response was a call to search_repo, with a query that would have been reasonable if that tool existed. It did not. A strict runtime returned an error: unknown tool. The model’s next message apologized and called read_file, which did exist, on a path it guessed. The guess was the argument problem. The first beat was invention.

On a looser harness I tried the same prompt, the search_repo text was shown to me as the assistant “deciding to search,” and nothing executed. I almost logged that as a refusal. It was not. The model had asked for a tool. The harness had dropped it on the floor and let the prose stand. The next person to wrap that model with a friendlier executor, one that shells out when it sees a verb, would run the invention. I count the raw message, not the polite UI.

A third run, same weights, called edit_file correctly by name and then, in the same turn, described a follow-up call to run_tests. There was no test tool. The description was future tense, which is how invention hides. Some parsers take only a fenced block. Some take any JSON object in the message. Know which you have. A sentence about run_tests is harmless until a parser is greedy.

What refusal would have looked like

Refusal is the model saying it cannot do the step with the tools it was given, and then stopping. No extra name. No JSON. No “I would call.” If the task needs a search and search is not on the list, the honest output is that the task cannot be done, or a read of a specific file you already named. Local models skip that honesty because they are trained to be helpful, and a helpful completion of a tool-shaped prompt is a tool call. The closest name in their memory wins over the closed list in the prompt, especially when the list is long or the names are similar.

Frontier models invent too. They do it less often on a short list, and they recover when the runtime says unknown. Local models often apologize and invent a different unknown name, or they repeat the same one. I do not wait for them to learn the list mid-chat. I keep the list short and I make unknown a hard error the loop cannot talk past. One error, then a human. A retry that lets the model pick a new name is how search_repo becomes grep, then find, then a shell command it composed.

The runtime is the part that makes invention safe or not

A good runtime has an allow-list and nothing else. The name must match. Extra fields are dropped or rejected. There is no fallback that runs “the closest tool.” There is no path where the assistant’s prose is executed as a command because it started with a verb. If your local stack logs the model output and also has a general shell tool on the same agent, an invented name is one bug away from a command. Take the shell off if the model is local and the list is supposed to be files. Invention plus a shell is not a research curiosity. It is a command the model wrote.

I test this on purpose with a boring prompt that asks for a tool I did not register. I want the log to show the unknown name and a rejection, and I want the workspace unchanged. If the workspace changes, the runtime is executing something it should not. If the log only shows a friendly “I can’t,” I look at the raw output. The friendly line may be the harness. The raw line may contain the invented call. I have been fooled by the friendly line once. I do not want a second time, so the test stays.

Names it likes to invent

Search, when you gave it read. Run, test, or shell, when you gave it edit. Browse or http_get, when you gave it files only. Send, when the task said tell someone and you gave it a draft file. These are not exotic. They are the next verb in the task. A local model completes the verb. Your job is to make sure the verb has no implementation unless you wrote it down.

Similar names are the other pattern. read_file versus read, edit versus write, list versus ls. The model emits the short one. A runtime that prefix-matches will run the wrong tool and call it a success. I require exact match. Prefix match is a feature that turns invention into execution. Turn it off.

Arguments to a real tool can also be invented in a way that resembles a new tool: a path that is a shell pipeline, a flag the schema does not have. Reject unknown fields. Do not ignore them. Ignoring a field named command is how a file tool grows a shell. I would rather the call fail.

What I change in the setup

Short exact allow-list. Unknown name is a hard stop, not a chat. No shell on the same agent as a small local model. Raw trace kept. A fixture prompt that asks for a missing tool, rerun when I change model or harness. If the fixture’s invented name ever changes the disk, I do not use that harness. If the model only ever refuses, I am pleasantly surprised and I still keep the allow-list, because the next model file I download will be chatty again.

I do not try to prompt my way to refusal as the only control. “Do not call tools that are not listed” is a sentence the model has already violated by calling search_repo. The list is in the prompt and it invented anyway. The parser is the control. The prompt is a wish.

A bigger local model does not retire the allow-list

A larger model on the same machine invents less often and still invents. I have watched a bigger instruct model stick to edit_file for two calls and then emit a test runner it remembered from training when the edit “should” be checked. The quality of the edit improved. The closed list did not become a law. If you bought the larger model so you could relax the parser, you bought the wrong thing. Spend the memory on better arguments. Keep the rejection. The day you swap weights, the fixture prompt should still fail closed. I rerun it as part of the swap, before I point the new weights at a real directory. A model file is not a harness. The harness is what you tested. The weights will surprise you once, and once is enough if a shell is listening.

Chat templates matter here. Some local runtimes wrap the prompt in a format that encourages tool JSON even when you wanted a plain answer. If the template says the assistant is a tool caller, you will get tool calls, including invented ones, on questions that did not need a tool. Use a template that matches the model, and still reject unknown names. I do not disable tools to “be safe” and leave a shell in the same process for me to run by hand while the model can also reach it. Separate the processes. The model’s stdout is untrusted text. It becomes a call only after the allow-list. That one sentence is the whole design. Everything else is how people talk themselves into prefix match.

The behavior to expect

Expect invention. Treat a clean refusal as a bonus you still verify in the raw output. A local model that “doesn’t use tools” may be a harness dropping calls, or a model that is about to name one you did not write. Make unknown names die, take away any general executor, and read the trace. The tool that does not exist should not be able to become the tool that runs. If the trace cannot show you the rejected name, you do not yet know which of the two failures you have, and the quiet one is the invented call.