Fixing silent tool-call failures caused by Ollama's 4,096-token default context

What they built

orca forge's app passes 29 tools to a model served by Ollama and expected a 16,384-token context window. Tools weren't being invoked and responses were cut off midway, and none of the usual signals explained why.

The app had earlier learned to react to a 400 error when the context was exceeded by trimming tools and old results. This time no 400 ever arrived, so that recovery never started, and the investigation ended up in how the request path set the context window.

Setup

• A local model served by Ollama that reports a 262,144-token context in its model info
• An app that estimates the window as the smaller of the model info and 16,384, and sends 29 tool definitions

How it works

1. Requests went through the OpenAI-compatible /v1/chat/completions endpoint, which didn't apply the expected window when loading the model, so it ran at the default 4,096.
2. When input exceeded the window, Ollama truncated it from the beginning instead of returning a 400.
3. The fix sends tool calls to /api/chat with options.num_ctx set to 16,384, converting arguments and history to that format, and avoids tool_choice: required, which Ollama doesn't support.

Results & numbers

• After the change there were zero 400 errors, all 29 tools were passed, and logs showed the full flow from tool call through search and retrieval to the final response.
• The author notes this fixes window handling, not the model's answer quality or speed.

Gotchas

• The context length in model info and the context actually used at load time are not necessarily the same.
• The absence of an error is not evidence that the whole input arrived.
• Moving only regular chat to the new route and leaving tool calls on the old one would keep the bug.

“The absence of an error was not evidence that the entire input had arrived.”— orca forge

Do it yourself

1. Check the context length in your model's info.
2. Check the context_length Ollama actually uses when the model loads.
3. Send tool calls to /api/chat and set options.num_ctx explicitly.
4. Don't rely on tool_choice: required with Ollama.
5. After the change, track the 400 count and confirm logs show tool calls reaching a final response.