Gemma 4 combined with Ollama delivers the first local coding-agent stack that is genuinely usable for real work on a laptop — no API key, no cloud round-trip, and no data leaving your machine. The 12B variant runs in roughly 7 GB of VRAM, scores 72% on LiveCodeBench v6, and supports native function calling so it can read files, run commands, and complete multi-step tasks. If you tried local coding agents a year ago and gave up because the model was too weak or tool calling kept breaking, the combination of Gemma 4's architecture and Ollama's reliability patches is what changed. For a broader look at running open-source models on your own hardware, see our local AI setup guide.
Last verified: 2026-07-30 · Best for laptops: Gemma 4 12B · Best for workstations: Gemma 4 31B · Best for edge/mobile: Gemma 4 E4B · Ollama 0.32+ required for tool-calling fixes · Pricing/limits change often — re-check before deploying.
What are Gemma 4 and Ollama, and how do they differ?
Gemma 4 is the model — a family of open-weight AI models released by Google DeepMind on April 2, 2026 under the Apache 2.0 license. Ollama is the engine — a local runtime that downloads and serves those models on your own hardware via a localhost API. You install Ollama, pull a Gemma 4 model, and your applications talk to it at http://localhost:11434 exactly as they would talk to OpenAI's API. Everything stays on your machine. (Google AI — Gemma 4 model card; Ollama — gemma4 library page)
The distinction matters because people often conflate the two. You can run Gemma 4 with other engines (vLLM, llama.cpp, LM Studio), and Ollama can run other models (Qwen, Llama, Mistral). But Ollama is the simplest path for most developers, and Gemma 4 is the model family that made local agentic coding practical.
Why does Gemma 4 matter for coding agents specifically?
Gemma 4 introduced three capabilities that turn a chat model into a working agent:
Native function calling. The model can generate structured tool-call objects — invoking functions, reading files, running shell commands — instead of merely describing what you should do. This is the mechanism that lets an agent act, not just talk. (Google AI — Function calling with Gemma 4)
Configurable thinking modes. Each model ships with a reasoning mode you can toggle on for hard architecture problems and off for quick edits. Thinking mode lifts coding and math benchmark scores by roughly 15–25 percentage points. (Google DeepMind — Gemma 4)
Native system prompt support. Gemma 4 handles the
systemrole directly, so you can set your stack, conventions, and guardrails once at the start of a session instead of restating them every turn. (Ollama — gemma4)
The model family is also multimodal — accepting text and image input — which means a coding agent can screenshot a UI, read an error dialog, or parse a diagram and act on what it sees.
Which Gemma 4 model should you pull?
Pick the largest model your hardware comfortably runs. Starting too big is the most common beginner mistake — everything feels sluggish and you blame the model when your hardware is simply tapped out.
| Model | Ollama tag | Parameters | Disk size (Q4) | Context window | Best for | VRAM needed |
|---|---|---|---|---|---|---|
| Gemma 4 E2B | gemma4:e2b |
2.3B effective / 5.1B total | ~4 GB | 128K | IoT, Android phones | ~2 GB |
| Gemma 4 E4B | gemma4:e4b |
4.5B effective / 8B total | ~6 GB | 128K | Laptops, edge devices | ~4–8 GB |
| Gemma 4 12B | gemma4:12b |
11.95B dense | ~7.6 GB | 256K | Most laptop developers | ~16 GB |
| Gemma 4 26B A4B | gemma4:26b |
25.2B total / 3.8B active (MoE) | ~18 GB | 256K | Mid-range desktop GPUs | ~16–24 GB |
| Gemma 4 31B | gemma4:31b (or :latest) |
30.7B dense | ~20 GB | 256K | Workstations, servers | ~64 GB |
Sources: Google AI — Gemma 4 model card; Ollama — gemma4; HuggingFace — google/gemma-4-12B-it.
The sweet spot for most developers is the 12B. It is smaller on disk than the default :latest (31B), offers double the context window of the edge models (256K vs 128K), and its benchmark scores land between the E4B and the 26B MoE — close enough to the MoE on reasoning that the gap rarely matters for daily coding work. On a 16 GB Mac or a laptop with a 16 GB GPU, it runs comfortably.
The 26B MoE (Mixture-of-Experts) is the efficiency pick: it holds 26 billion total parameters but activates only about 3.8 billion per token, so it runs nearly as fast as a 4B model while reasoning nearly as well as the 31B dense flagship. (gemma4all.com — Gemma 4 benchmarks)
How strong is Gemma 4 at coding, really? (Benchmark reality check)
The jump from Gemma 3 to Gemma 4 on coding benchmarks is not incremental — it is a generational shift:
| Benchmark | Gemma 3 27B | Gemma 4 E4B | Gemma 4 12B | Gemma 4 26B A4B | Gemma 4 31B |
|---|---|---|---|---|---|
| MMLU-Pro | 67.6% | 69.4% | 77.2% | 82.6% | 85.2% |
| LiveCodeBench v6 | 29.1% | 52.0% | 72.0% | 77.1% | 80.0% |
| Codeforces Elo | 110 | 940 | 1659 | 1718 | 2150 |
| AIME 2026 (no tools) | 20.8% | 42.5% | 77.5% | 88.3% | 89.2% |
| GPQA Diamond | 42.4% | 58.6% | 78.8% | 82.3% | 84.3% |
Sources: Google AI — Gemma 4 model card; HuggingFace — google/gemma-4-12B-it model card; gemma4all.com — Gemma 4 benchmarks.
What this means in practice: Gemma 4 31B's Codeforces Elo of 2150 places it at "Candidate Master" level — a rank fewer than 5% of competitive programmers reach. LiveCodeBench v6 uses problems published after the model's training cutoff, so the 80% score reflects genuine coding ability, not memorization. Even the 12B at 72% and the E4B at 52% crossed the threshold from "occasionally helpful" to "reliably useful" for code generation — the E4B outscored the entire previous generation's flagship (Gemma 3 27B at 29.1%).
One caveat on long context: a model accepting 256K tokens does not use all of them equally well. On the MRCR v2 8-needle retrieval test at 128K context, the 31B scored 66.4% and the 26B scored 44.1%. Keep your agent sessions focused — stuff the context with what matters, not everything you can find.
How do you set up a Gemma 4 coding agent with Ollama?
This is the complete setup, start to finish, in seven steps.
Step 1: Install Ollama (version 0.32 or later)
Download from ollama.com (macOS/Windows installers, or the Linux one-liner curl -fsSL https://ollama.com/install.sh | sh). Verify it is serving:
ollama --version
curl http://localhost:11434
The second command should return "Ollama is running." That local HTTP endpoint on port 11434 is what your editor extensions and agent scripts will talk to.
Critical: You need Ollama 0.32 or later for the Gemma 4 tool-calling reliability fixes. If you are on an older version and your agent keeps failing to call tools, that is your problem — update before blaming the model. (GitHub — Ollama releases)
Step 2: Pull a Gemma 4 model for your hardware
# 16 GB VRAM — best for most laptops
ollama pull gemma4:12b
# 8 GB VRAM — edge model, still capable
ollama pull gemma4:e4b
# 24 GB+ VRAM — MoE efficiency pick
ollama pull gemma4:26b
# 64 GB+ VRAM — flagship dense
ollama pull gemma4:31b
Test it before wiring up an agent:
ollama run gemma4:12b "Write a Python function that merges two sorted lists"
If tokens stream at a comfortable reading pace, you are good.
Step 3: Install an agent-capable editor extension
Two options cover the full range from assist to autonomy:
- Continue (VS Code / JetBrains) — ghost-text autocomplete + chat + inline edits. Install first; it replaces GitHub Copilot.
- Cline (VS Code) — autonomous agent for multi-file tasks: it plans, edits across files, runs commands, and shows you diffs to approve. Better with bigger models; overkill for simple completion.
Both support Ollama as a provider. Point them at http://localhost:11434.
Step 4: Configure the model in Continue
Open Continue's config (config.yaml) and register the model:
models:
- name: Gemma 4 12B (local)
provider: ollama
model: gemma4:12b
roles: [chat, edit, apply]
That is the whole integration: completions as you type, Cmd/Ctrl+L to chat about selected code, Cmd/Ctrl+I for inline edits.
Step 5: Set a system prompt (use Gemma 4's native system role)
Gemma 4 supports the system role directly — take advantage of it. Tell the model your tech stack, your coding conventions, and what it should never do. In Ollama, create a Modelfile:
FROM gemma4:12b
SYSTEM """
You are a senior software engineer working on a Next.js 16 + .NET 9 project.
Follow the existing code style. Never delete code you do not understand.
Always explain changes in 2-3 sentences before making them.
Prefer small, testable edits over large rewrites.
"""
Build it:
ollama create my-coder -f Modelfile
Now run ollama run my-coder or point Continue at model: my-coder. The system prompt persists for the entire session.
Step 6: Raise the context window
Ollama defaults to a small context window, and coding agents eat context fast — system prompt + file contents + tool schemas + diffs add up. Raise it:
OLLAMA_CONTEXT_LENGTH=16384 ollama serve
Or set num_ctx in your Modelfile. 16K is a good floor for agent work; go higher only if your VRAM allows, since KV cache grows with context length.
Step 7: Verify the tool-calling loop
Real test: open a project, ask Continue to explain a function, accept a couple of completions, then give Cline a small task like "add input validation to this endpoint and update its test." If completions feel sluggish, drop to a smaller model or lower quant. If agent edits go off the rails, that is usually the model being too small for the task — not a tooling bug.
For a function-calling smoke test from the command line, use Ollama's Python library:
from ollama import chat
response = chat(
model='gemma4:12b',
messages=[{
'role': 'user',
'content': 'What files are in the current directory? Use the ls tool.'
}],
tools=[{
'type': 'function',
'function': {
'name': 'run_shell',
'description': 'Run a shell command and return stdout',
'parameters': {
'type': 'object',
'properties': {
'command': {'type': 'string', 'description': 'The shell command to run'}
},
'required': ['command']
}
}
}],
)
print(response.message.tool_calls)
If you get a structured tool-call object back with the correct command, tool calling is working. If you get prose instead of a call, check your Ollama version and re-pull the model — stale caches are the number one reason developers miss template and tool-call fixes.
How do you use Gemma 4's thinking mode for coding tasks?
Gemma 4's thinking mode runs an internal chain-of-thought before generating the visible answer. Turn it on for hard problems; turn it off for speed.
When to use thinking mode (on):
- Architecture decisions ("Should this service be split into two?")
- Debugging complex logic ("Why is this function returning null under load?")
- Multi-file refactoring plans
- Algorithm design
When to skip thinking mode (off):
- Quick syntax fixes
- Boilerplate generation
- Simple completions
- Any task where latency matters more than depth
In Ollama, you can toggle thinking in the chat template:
response = chat(
model='gemma4:12b',
messages=[{'role': 'user', 'content': 'Design a caching layer for a .NET API that handles 10K req/s'}],
think=True, # thinking mode on
)
Per Google's documentation, thinking mode lifts coding and math benchmark scores by approximately 15–25 percentage points — the 31B's LiveCodeBench and AIME scores are reported with thinking enabled. (Google DeepMind — Gemma 4; Google AI — Gemma 4 model card)
What changed in Ollama's tool-calling, and why does it matter?
Ollama 0.32 shipped significant improvements to Gemma 4's tool-calling reliability — the specific fix that separates an agent that finishes a task from one that gets stuck halfway through.
Tool calling is the bridge between a language model and the outside world. Without reliable tool calling, an agent might describe what it would do but never actually do it. With it, the model can:
- Call a function to read a file
- Run a shell command and parse the output
- Search a codebase
- Execute a test suite
- Write and save a file
If you tried local coding agents before and gave up because the agent kept failing to invoke tools or returned garbled function-call JSON, this is the problem that got worked on. (GitHub — Ollama releases; Google AI — Function calling with Gemma 4)
A July 2026 stealth update to Gemma 4 itself further patched tool-calling bugs and truncated responses — Google reported agentic benchmark gains of up to +10.1% (Tau2 Telecom) on the 31B model. Re-pull your models after updating Ollama to pick up these fixes. (The Decoder — Gemma 4 stealth update)
Gemma 4 vs other local coding models: how does it compare?
Gemma 4 is not the only option for local coding agents, and the right pick depends on your priorities.
| Dimension | Gemma 4 12B | Qwen 3.6 27B | Gemma 4 E4B |
|---|---|---|---|
| License | Apache 2.0 | Apache 2.0 | Apache 2.0 |
| VRAM needed | ~16 GB | ~24–48 GB (Q8 on 48 GB Mac) | ~4–8 GB |
| Multimodal | Text + image + audio + video | Text-first (vision variants separate) | Text + image + audio + video |
| Best for | Multimodal coding agents, laptops | Pure text coding, instruction-following | Edge, privacy-bound tasks |
| LiveCodeBench v6 | 72.0% | Community favorite (run your own eval) | 52.0% |
| Tool calling | Native, patched in Ollama 0.32 | Strong on coding agents | Native, improving |
Sources: gemma4all.com — Gemma 4 benchmarks; Ollama — gemma4; explainx.ai — Gemma 4 July 2026 update.
Practical routing:
- Text-only coding agent on a powerful Mac/PC → start with Qwen 3.6 27B (community favorite for instruction-following)
- Screenshot / document / UI agent or laptop-bound → Gemma 4 12B (multimodal + fits 16 GB)
- On-device / edge / mobile → Gemma 4 E4B (runs on 4 GB RAM)
- Server-class workstation → Gemma 4 31B (best quality, needs ~64 GB)
What does a local coding agent cost to run?
After hardware, effectively just electricity. A 16 GB GPU coding all day costs a few dollars per month in power, versus $10/month for Copilot Pro or $20/month for Cursor Pro indefinitely.
If you already own the GPU (or your Mac has enough unified memory), the payback is immediate — the models are free under Apache 2.0. If you do not, a used RTX 4060 (8 GB VRAM, ~$200) runs the E4B comfortably; a 16 GB GPU or a 16 GB+ Mac runs the 12B. The 31B dense flagship needs roughly 64 GB of VRAM or system RAM with offloading, making it a workstation play.
The real cost is time: getting the setup working the first time, debugging agent loops, and calibrating your system prompt. Budget an afternoon for a first successful setup rather than expecting it in 10 minutes.
What this means for you
If you are a developer or small team that values privacy (client work under NDA, regulated industries, unreleased products) or simply wants zero recurring AI costs, a local coding agent with Gemma 4 and Ollama is now a practical reality. Start with the 12B on a 16 GB machine, use Ollama 0.32+, set a system prompt, and keep sessions focused. You get Copilot-quality autocomplete and multi-file agent edits with nothing leaving your hardware. Want to go further and build a full autonomous agent stack? See our open-source AI agent stack guide and our walkthrough on building a free local AI agent with Gemma 4.
If you are already paying for a cloud coding assistant and it works for you, there is no urgent reason to switch — the cloud models are still ahead on the hardest tasks. But the gap has closed enough that "local is too weak for real work" is no longer true, and the privacy, cost, and offline advantages are real.
FAQ
Q: Can Gemma 4 actually run coding agents on a regular laptop? A: Yes. The Gemma 4 12B model runs in roughly 7.6 GB of disk space and 16 GB of VRAM/unified memory, scoring 72% on LiveCodeBench v6 and 77.2% on MMLU-Pro — both well above the previous generation's 27B flagship. With Ollama 0.32's tool-calling fixes, it can reliably call functions, read files, and complete multi-step coding tasks on a standard 16 GB laptop.
Q: What is the difference between Gemma 4 and Ollama? A: Gemma 4 is the AI model — a family of open-weight models from Google DeepMind. Ollama is the local runtime that downloads and serves models on your hardware via a localhost API at port 11434. You use Ollama to run Gemma 4 (or other models) locally.
Q: Which Gemma 4 model size should I start with?
A: Start with the 12B (ollama pull gemma4:12b) if you have 16 GB of VRAM or unified memory. If you have only 8 GB, start with the E4B (gemma4:e4b). Move up to the 26B MoE or 31B only when the smaller model feels limiting — never start with the biggest, or everything feels slow and you will blame the model when your hardware is the bottleneck.
Q: Do I need Ollama 0.32 specifically for Gemma 4 tool calling? A: Yes. Ollama 0.32 includes significant improvements to Gemma 4's tool-calling reliability. If you are on an older version and your agent fails to call tools or returns garbled JSON, updating Ollama — not swapping models — is the fix. Re-pull the model after updating to pick up the latest weight and template patches.
Q: Does any of my code leave my machine with this setup? A: No. Ollama serves on localhost, and with your editor extension pointed at it, prompts, code context, and completions all stay on your hardware. This is the core argument for local coding agents in privacy-sensitive codebases — NDA work, regulated industries, or anything you do not want sent to a third-party API.
Q: Is Gemma 4 free for commercial use? A: Yes. Gemma 4 is released under the Apache 2.0 license, which permits commercial use, modification, and redistribution without asking permission. (Google AI — Gemma 4 model card)
Discussion
0 comments