Verdict: The Ollama v0.32.1 release (July 16, 2026) is the update that finally makes Gemma 4 usable as the brain of a local, multi-step AI agent. It fixes the specific failure that kept breaking real workflows — the model would call a tool, get the result back, and then stall or drop the thread instead of continuing. Combined with the ~90% inference speedup from multi-token prediction shipped three weeks earlier, a fully local Gemma 4 agent can now plan a task, call tools, read results, and finish the job on your own laptop — without the half-finished answers that used to force you to babysit every step.
Last verified: 2026-07-29 · Gemma 4 31B scores 86.4% on the τ2-bench Retail agentic tool-use benchmark (vs 6.6% for Gemma 3 27B) · Ollama v0.32.1 shipped the tool-response continuation fix on July 16, 2026 · MTP landed in v0.31.1 (June 30, 2026) and is on by default on Apple Silicon.
- The fix: "more reliable tool-response continuations" — Gemma 4 now finishes a multi-step tool chain instead of stopping after the first call.
- The speedup: multi-token prediction makes Gemma 4 generate tokens nearly 90% faster on the Aider polyglot coding benchmark on Apple Silicon (on by default, no config).
- The stability patch: fixed a recurrent MLX model cache leak that could grow memory use across requests.
- The convenience: the interactive agent now receives your current working directory for better project context.
Pricing/limits note: Ollama and Gemma 4 are both free, Apache 2.0-licensed — no per-call cost and no monthly active-user restriction.
What Is Ollama's Gemma 4 Tool Calling Fix (v0.32.1)?
Ollama v0.32.1 — released July 16, 2026 — patches the Gemini-style function-calling loop that Gemma 4 uses when you hand it tools. The release notes describe it as "improved Gemma 4 tool calling and multi-turn reasoning, including more reliable tool-response continuations" (Ollama GitHub releases). In practice, this fixes the moment where a local agent would call a function, receive the output, and then refuse to keep going — or claim it had finished a job it never ran.
This single continuation bug was the reason local Gemma 4 agents felt unreliable even after the model's raw agentic score jumped from 6.6% (Gemma 3 27B) to 86.4% (Gemma 4 31B) on the τ2-bench Retail benchmark, per Google DeepMind's own published numbers (Gemma 4 model page). The model could call tools — it just couldn't always follow through after.
Why tool calling kept breaking before this update
Tool calling in an LLM agent is a three-step handshake:
- The model decides a tool is needed and emits a structured
tool_callsblock (a JSON instruction naming the function and its arguments). - Your application executes the function and returns the result as a
role: "tool"message. - The model reads that result and either calls the next tool or writes its final answer.
Step 3 is where it broke. On Gemma 4, the parser that recognized tool-call output sometimes misread the result message, or the model would emit a half-formed tool call and then stop. Ollama's own GitHub issue tracker documented early "gemma4 tool call parsing failed" warnings even after the initial v0.20.x fixes, particularly when tool arguments contained characters needing JSON escaping — shell commands, regex patterns, multi-line code (ollama/ollama#15315). The v0.32.1 continuation patch targets exactly that weak spot: the model now reliably reads the tool result back and keeps the chain alive.
How Much Faster Is Gemma 4 on Ollama Now (Multi-Token Prediction)?
Gemma 4 on Ollama is now nearly 90% faster at generating tokens on Apple Silicon, on average across a real coding-agent benchmark — measured on the Aider polyglot benchmark using Gemma 4 12B in its nvfp4 quantization on an M5 Max (Ollama blog: Faster Gemma 4 on MLX with multi-token prediction). The speedup landed in Ollama v0.31.1 on June 30, 2026 and is on by default with no configuration needed.
What multi-token prediction actually does: Gemma 4 ships with a small draft model that runs alongside the main model and proposes the next several tokens in one pass. The main model verifies the proposal in a single step and keeps the tokens it agrees with — so when the draft is correct (which it often is for code, where closing brackets and repeated identifiers are predictable), the model commits several tokens for the cost of one forward pass.
This matters most for coding agents and tool loops, where the model is called repeatedly as it reads files, runs tools, and works through a task. At a conservative ~50 tok/s baseline without MTP, a 5-turn agent loop producing 1,000 tokens per turn takes about 100 seconds end-to-end; at the post-MTP ~90 tok/s rate the same loop takes ~56 seconds — the difference between a tool that breaks your flow and one that maintains it.
| Scenario | Without MTP (~50 tok/s) | With MTP (~90 tok/s) | Source |
|---|---|---|---|
| 1,000-token agent turn | ~20 sec | ~11 sec | Aider polyglot, Ollama |
| 3,000-token code review | ~60 sec | ~33 sec | Community-reported throughput |
| 5-turn agent loop (1k tokens each) | ~100 sec | ~56 sec | Derived |
Watch your model tag: pull
gemma4:12b-mlxon Mac, not the plaingemma4:12b. The-mlxvariant uses Apple's MLX engine and gets MTP; the default tag goes through the standard llama.cpp path without it.
How Do You Set Up Gemma 4 Tool Calling With Ollama (Step by Step)?
Here's how to get the reliability-fixed tool-calling loop running on your own machine. These steps assume Ollama 0.32.1 or newer — update first if you're on anything older.
Step 1 — Update Ollama to v0.32.1 or newer.
On macOS: brew upgrade ollama (or download the latest .dmg from ollama.com). On Linux: curl -fsSL https://ollama.com/install.sh | sh. On Windows: download the .exe installer from ollama.com. Confirm with ollama --version — you want 0.32.1 or higher.
Step 2 — Pull the Gemma 4 variant that fits your hardware.
ollama pull gemma4:12b-mlx # Apple Silicon, best speed with MTP
ollama pull gemma4:26b # 24 GB GPU (RTX 3090/4090) or 32 GB Mac
ollama pull gemma4:31b # dense flagship, 24 GB+ VRAM or 64 GB Mac
ollama pull gemma4:e4b # default, 9.6 GB, fits 16 GB Macs
For agent workloads on Apple Silicon, gemma4:12b-mlx is the sweet spot — it gets multi-token prediction and is the variant Ollama benchmarked for the 90% speedup.
Step 3 — Override the silent 4K context default.
Ollama defaults to a 4,096-token context window regardless of the model's support, and Gemma 4 supports 128K–256K. Set num_ctx explicitly or you'll silently truncate agent prompts:
ollama run gemma4:12b-mlx
>>> /set parameter num_ctx 32768
For API calls, pass it in options: {"num_ctx": 32768}.
Step 4 — Define your tools and run the chat loop.
Send tools as JSON-schema objects in the tools field. When the model decides to use one, the response contains tool_calls instead of content; you execute the function, return the result as a role: "tool" message, and the model continues to its final answer. The v0.32.1 fix is what makes that "and the model continues" part reliable.
import json, urllib.request
def call_ollama(payload: dict) -> dict:
data = json.dumps(payload).encode("utf-8")
req = urllib.request.Request(
"http://localhost:11434/api/chat",
data=data,
headers={"Content-Type": "application/json"},
)
with urllib.request.urlopen(req) as resp:
return json.loads(resp.read().decode("utf-8"))
tools = [{
"type": "function",
"function": {
"name": "read_file",
"description": "Read the contents of a file.",
"parameters": {
"type": "object",
"properties": {"path": {"type": "string"}},
"required": ["path"],
},
}
}]
resp = call_ollama({
"model": "gemma4:12b-mlx",
"messages": [{"role": "user", "content": "Read the README and summarize it."}],
"tools": tools,
"stream": False,
"options": {"num_ctx": 32768},
})
# resp["message"]["tool_calls"] now contains the function call to execute.
# Run it, append {"role":"tool","content":<result>} to messages, and call again.
Step 5 — Use the interactive agent for one-off tasks (no code needed).
Since v0.32.0 (July 11, 2026), running bare ollama in your terminal launches an interactive agent that can chat, write code, search the web, and delegate work right from the prompt. The v0.32.1 patch makes that agent receive your current working directory automatically, so it stops mixing up files when working across multiple paths (Ollama v0.32.0 release notes).
What Changed in Ollama v0.32.1 (Full Breakdown)?
| Change | What it fixes | Who it helps |
|---|---|---|
| Improved Gemma 4 tool-response continuation | Agents stopping mid-chain after a tool result | Anyone running a local agent that calls functions |
| Fixed recurrent MLX model cache leak | Memory growing across requests until slowdown or crash | Long-running Mac sessions |
OLLAMA_LOAD_TIMEOUT respected for MLX text models |
Slow model loads being cut off prematurely | Mac users with large models |
| Interactive agent receives current working directory | File mix-ups when working across multiple folders | Terminal-first workflows |
ollama launch model-picker fix for deprecated --model |
Picker not opening for deprecated models | Anyone with older tags |
Source: Ollama v0.32.1 release notes.
The memory-leak fix is easy to overlook but matters for anyone running Ollama as a persistent service. Before the patch, the MLX cache could slowly grow across requests until the process slowed or crashed — which is a real problem when you have an agent running in the background all day.
Which Gemma 4 Variant Should You Pick for Tool Calling?
Tool calling ships natively in every Gemma 4 size, but the practical choice depends on your hardware and whether you want the MTP speedup:
| Variant | Ollama tag | Min VRAM/RAM | MTP on Mac? | Best for |
|---|---|---|---|---|
| E2B | gemma4:e2b |
4 GB | No (text only) | Phones, Pi 5, edge |
| E4B (default) | gemma4:e4b |
8 GB | No | 16 GB Macs, laptops |
| 12B | gemma4:12b-mlx |
16 GB | Yes | Coding/agent work on Apple Silicon |
| 26B MoE (~4B active) | gemma4:26b |
24 GB | Yes (-mlx tag) | RTX 3090/4090, 32 GB Mac |
| 31B Dense | gemma4:31b |
24 GB+ | Yes (-mlx tag) | Max quality, 64 GB Mac / multi-GPU |
For agentic tool use specifically, the 12B MTP variant on Apple Silicon gives the best responsiveness-per-dollar (it's free). For raw agentic quality, the 31B Dense holds the 86.4% τ2-bench score — but it needs a 24 GB+ GPU or a 64 GB Mac and runs roughly 3–4x slower than the 26B MoE on the same card (Gemma 4 model card).
What's the Difference Between Gemma 4 and Gemma 3 for Tool Calling?
Google's own benchmark shows the jump from Gemma 3 to Gemma 4 on agentic tool use was the largest single-generation improvement in the Gemma line — and it's the reason a local agent built on Gemma 4 is now viable where the same agent on Gemma 3 was essentially unusable.
| Capability | Gemma 3 27B IT | Gemma 4 31B IT | Source |
|---|---|---|---|
| τ2-bench Retail (agentic tool use) | 6.6% | 86.4% | Google DeepMind |
| LiveCodeBench v6 (coding) | 29.1% | 80.0% | Google DeepMind |
| AIME 2026 (math, no tools) | 20.8% | 89.2% | Google DeepMind |
| MMMLU (multilingual, no tools) | 67.6% | 85.2% | Google DeepMind |
| Arena AI text (ELO, Apr 2026) | 1365 | 1452 | Google DeepMind |
| Vision/Audio input | No | Yes | Google DeepMind |
The 6.6% → 86.4% jump on τ2-bench is the number that explains why the Ollama continuation fix matters at all: Gemma 3 wasn't smart enough to use tools reliably even when the parsing worked, while Gemma 4 is smart enough that the parsing/continuation bug was the only thing holding it back.
What This Means for You
If you write code (even a little): Pull gemma4:12b-mlx through Ollama on a Mac, update to v0.32.1, and test it on a real file you're working on — not a demo prompt. Give it a messy script, ask it to find slow spots, rewrite them, run the tests, and tell you what happened. The continuation fix is the difference between the agent stopping after "I found it" and the agent actually handing you back a finished, tested result. For a deeper walk-through of building a free local agent on Gemma 4, see our free local AI agent guide with Gemma 4 and Hermes.
If you run a small business: The use case is the same pattern the continuation fix unlocks everywhere — an AI that can carry a multi-step task through to the end without you checking it after every step. Write the signup form, check every field works, fix any mistakes, and tell you when it's done — in one pass. Some concrete places to start are in our guide on setting up local AI to run open-source models on your own hardware.
If you build agent stacks: Gemma 4's native function calling and 256K context make it a solid open brain for an autonomous agent — but the continuation fix is the one you had to wait for. The open-source autonomous-agent-stack pattern we covered in our open-source AI model autonomous agent guide is now actually deployable on Gemma 4 where it stalled before. And if you're designing the loop itself, our guide to AI agent loops with loop engineering covers the doer-judge verification pattern that prevents exactly the "AI says it finished but never did" failure this patch addresses.
The honest caveat: Every time local tool use gets more reliable, it means less manual checking — which is good for throughput but also means the trusted tool is doing more without you watching as closely. The fix for that isn't to slow down; it's to run a verification step (a doer-judge loop, a test suite, a human checkpoint) at the end of every agent chain so you're using the fix on purpose, not just letting it run blind. See our doer-judge loop field guide for the exact pattern.
FAQ
Q: What does "tool-response continuation" mean in plain language? A: It means after the AI calls a tool and gets the result back, it keeps going with the task instead of stopping, claiming it finished, or dropping the thread. Before the v0.32.1 fix, Gemma 4 would often emit a valid tool call, receive the output, and then halt — leaving you with a half-finished answer. The patch makes the model reliably read the tool result and continue to its next step or final answer.
Q: Which Ollama version has the Gemma 4 tool calling fix?
A: Ollama v0.32.1, released July 16, 2026. It contains "improved Gemma 4 tool calling and multi-turn reasoning, including more reliable tool-response continuations." Make sure ollama --version shows 0.32.1 or newer (Ollama release notes).
Q: Is multi-token prediction on by default, or do I have to configure it?
A: MTP is on by default on Apple Silicon with no configuration — Ollama auto-tunes how many tokens to draft as the model runs. The only thing you need to do is pull the -mlx variant (e.g., gemma4:12b-mlx) instead of the plain tag (Ollama MTP blog).
Q: How fast is Gemma 4 after the MTP speedup? A: Ollama's own measurement on the Aider polyglot benchmark showed Gemma 4 12B (nvfp4) generating tokens nearly 90% faster on an M5 Max. Community-reported throughput lands around ~90 tok/s post-MTP versus ~50 tok/s without it on comparable M-series hardware — roughly double the usable text output per second.
Q: Does the Ollama 0.32.1 tool calling fix work on Windows and Linux, or just Mac? A: The tool-response continuation fix is engine-agnostic and applies on every platform Ollama runs on — macOS, Linux, and Windows. The multi-token prediction speedup is currently Apple Silicon and MLX-only; developers on Linux or Windows with CUDA do not see the MTP benefit in v0.31/v0.32, though the speculative-decoding mechanism itself works on GPU hardware.
Q: How does Gemma 4's tool calling compare to Gemma 3? A: Google DeepMind reports τ2-bench Retail (agentic tool use) jumped from 6.6% on Gemma 3 27B to 86.4% on Gemma 4 31B — a 13x improvement that moved the model from "essentially unusable for agents" to "competitive at tool use." That's exactly why the Ollama continuation patch matters: the only thing holding a 86.4%-capable model back was the parsing/continuation bug.
Q: Do I need a GPU to run Gemma 4 tool calling locally?
A: No. The E2B and E4B variants run on 4–8 GB of RAM and can execute tool calls offline on a phone, a Raspberry Pi, or a 16 GB Mac with near-zero latency. For coding-agent workloads where you want MTP and faster loops, you want the 12B -mlx or 26B/31B variants — those benefit from 16–24 GB+ of VRAM or unified memory for comfortable headroom.
Discussion
0 comments