Yes, you can run a 26-billion-parameter language model on a Mac with just 2GB of RAM — and get usable output at 5–35 tokens per second. An open-source project called TurboFieldfare, released in July 2026 by iOS and Metal engineer Andrey Mikhaylov, proves it by running Google's Gemma 4 26B-A4B model on an 8GB M2 MacBook Air with only ~2GB of memory occupied.
The trick is not a smaller model or a math shortcut. It exploits a structural feature of Mixture-of-Experts (MoE) architectures: only a small fraction of the model's parameters are active for any given token. TurboFieldfare keeps the always-needed core (attention, router, embeddings, shared expert) resident in RAM — about 1.35GB — and streams the remaining expert weights from the SSD on demand, a few megabytes at a time. Apple Silicon's unified memory architecture means the GPU can read those bytes directly without a PCIe bus hop, which is what makes this approach viable on a Mac and impractical on a typical discrete-GPU PC.
For anyone building, researching, or just experimenting with local AI on resource-constrained hardware, this represents a genuine shift: the bottleneck moved from RAM capacity to SSD throughput, opening the door to large-model inference on machines that were previously locked out entirely.
Last verified: 2026-08-01 · Works on any Apple Silicon Mac with 8GB+ RAM · Requires macOS 26 + Metal 4 · 5–6 tok/s on M2 Air, 31–35 tok/s on M5 Pro · Open-source (Apache 2.0)
How Does a 26B Model Fit in 2GB of RAM?
The model never actually fits in 2GB. The insight is that it does not need to. Gemma 4 26B-A4B is a Mixture-of-Experts model with 128 expert networks per layer, but only 8 of those experts (plus one shared expert) activate for any single token. That means roughly 3.88 billion of the 25.2 billion total parameters do any work on a given forward pass — about 85% of the weight file is idle at any instant.
TurboFieldfare splits the model into two piles at install time:
Pile 1 — Always-resident (~1.35GB): The attention layers, the router, the embeddings, and the shared expert that runs on every token regardless of routing. These are memory-mapped and stay in RAM the entire time the model is loaded.
Pile 2 — Expert weights (~12.9GB on SSD): The 128 routed experts across 30 transformer layers (~3.36MB each). These never get fully loaded. They sit on the SSD and are pulled into memory a few megabytes at a time, on demand, using bounded parallel pread calls into Metal-visible buffers.
The result: a ~2GB memory footprint (weights + KV cache) instead of the ~14.4GB that a conventional 4-bit quantized load of Gemma 4 26B-A4B would require.
What Makes This Specific to Apple Silicon?
On a conventional PC with a discrete GPU, loading a weight from SSD into the GPU requires two memory copies and a bus traversal: read from SSD into system RAM, then push across the PCIe bus into the GPU's VRAM. Doing this thousands of times per second — once per layer per token — would tank performance and make the approach a non-starter.
Apple Silicon has unified memory: the CPU and GPU share the same physical RAM pool, so there is no VRAM to copy into. A Metal buffer is simply memory that both the CPU and GPU can see. The CPU reads bytes straight off the SSD into a buffer the GPU is about to use, skipping the PCIe hop entirely.
TurboFieldfare leans on this further with its file format. Normally, model weights on disk sit in a storage layout that needs to be unpacked and converted before a GPU can consume them. The installer rearranges the data into the exact byte layout the Metal kernel expects — right down to the 4-bit quantized values — without re-encoding. Reading the file is loading the weight. There is no conversion step in between.
What Happens During Token Generation?
Each token passes through 30 transformer layers in sequence. At every layer, the same dance happens:
- Attention runs on resident weights. The model looks back over the conversation so far. This step uses only the ~1.35GB already in memory — no disk access yet.
- The router selects 8 experts. The router, also resident, looks at the attention output and names 8 of 128 experts. This is where the constraint bites: you cannot know which experts you need until you have already done half the work on that layer. There is no reading ahead and no prefetching, because the choice depends on this token and every token before it.
- The CPU fetches missing experts from SSD. For each of the 8 selected experts, the system checks a 16-slot LFU (Least Frequently Used) cache kept in RAM per layer. If the expert is already cached, you get it instantly. If not, the CPU reads it from the SSD and evicts whichever cache entry has been used least often.
- The GPU works on the shared expert during the fetch. While the CPU is off reading from disk, the GPU computes the shared-expert branch that runs for every token no matter what the router picked. The disk read hides inside work the model had to do anyway — so you get most of the fetch for free.
- Combine and move to the next layer. Shared and routed outputs merge, and the token advances to the next layer, where the process repeats.
The LFU cache choice is deliberate. Some experts are picked constantly across many different tokens; others rarely activate. Counting how often an expert gets used keeps the popular ones around better than an LRU (Least Recently Used) strategy, which would evict based on recency alone. The design is a bet that expert routing is predictable — and in practice, with Gemma 4, it appears to be.
How Fast Is It in Practice?
Benchmark data from the project's repository, measured on real Apple Silicon hardware:
| Mac Model | Chip | RAM | Decode Speed | Memory Footprint |
|---|---|---|---|---|
| MacBook Air (M2) | M2 | 8 GB | 5.1–6.3 tok/s | ~1.9–2.1 GB |
| MacBook Pro (M5 Pro) | M5 Pro | 24 GB | 31–35 tok/s | ~2.1 GB |
For comparison, running the same model conventionally via mlx-lm on an M5 Pro (loading all 14.3GB into memory) gives 76–82 tok/s — but requires 8–10GB of RAM and full model residency.
What do those speeds feel like?
| Speed Range | Experience |
|---|---|
| 30+ tok/s | Feels instant — text appears faster than you can read it |
| 15–25 tok/s | Comfortable, faster than natural reading speed |
| 5–10 tok/s | Usable but you notice the wait — fine for batch tasks, code review, document Q&A |
| Below 5 tok/s | Painful for interactive chat; only acceptable for background generation |
The 5 tok/s on a base M2 Air is enough for genuinely useful local tasks: document summarization, code review, private Q&A against sensitive files, and batch text generation. On M4 and M5 hardware, the numbers cross into comfortable interactive territory — and the SSD throughput, not RAM size, becomes the primary bottleneck.
How Is This Different From Just Using Ollama or MLX?
Standard local LLM tools like Ollama, MLX, and LM Studio load the entire model into memory before generation begins. If the model does not fit, they either refuse to run or offload layers to system RAM over the PCIe bus with dramatic performance degradation.
The key distinction:
| Approach | What Loads | RAM Needed | Speed | Works on 8GB Mac? |
|---|---|---|---|---|
| Ollama / MLX (full load) | Entire model in RAM | ~14GB for 26B at 4-bit | 76–82 tok/s (M5 Pro) | No — 26B does not fit |
| llama.cpp CPU offload | Layers split RAM/CPU | ~5–8GB, rest on CPU | 1–3 tok/s (painful) | Technically yes, miserably |
| TurboFieldfare (SSD streaming) | Core resident, experts on SSD | ~2GB | 5–35 tok/s | Yes, genuinely usable |
TurboFieldfare is not a replacement for Ollama. It is a specialized technique for a specific situation: you want a large MoE model on a machine where it cannot fully fit in RAM. If your model fits comfortably, standard tools are faster.
Can You Do This on a Windows PC?
Partially, yes — but with a meaningful penalty. A Windows project inspired by the same technique streams MoE experts from NVMe SSD on consumer hardware, running a 32GB model (Qwen3-30B-A3B at Q8) on a PC with 12GB VRAM + 16GB RAM at 2.5–4.3 tok/s.
The reason it is slower: the GPU still needs expert weights delivered into VRAM, which means the CPU reads from SSD, then copies across the PCIe bus. On Apple Silicon, the CPU reads from SSD directly into a shared buffer the GPU can already see. That double-copy overhead is the fundamental difference unified memory makes.
The Windows approach works and is worth knowing about, but the architecture advantage Apple Silicon holds for this specific technique is real.
What Are the Trade-Offs and Limitations?
This is not a free lunch. The approach has honest costs that matter if you plan to use it heavily:
SSD wear. Pulling expert weights from SSD means continuous read traffic during generation. NVMe drives have rated endurance limits (TBW — terabytes written). Heavy, daily use of SSD-streamed inference puts more wear on the drive than a model loaded into RAM. For occasional or moderate use, this is negligible. For a 24/7 server workload, you may want to consider it.
Energy cost. A 2025 study from Seoul National University found that offloading MoE expert weights to SSDs increased per-token energy consumption by approximately 4.9× compared to keeping experts in HBM (high-bandwidth memory), and SSD reads accounted for ~80% of total per-token energy in that scenario. Prefetching and latency hiding help with speed but cannot eliminate this energy penalty. However, on a laptop running a few hundred tokens, the absolute energy cost is still modest — the concern scales with sustained server-grade workloads.
Single-model, single-platform. TurboFieldfare currently runs only the Gemma 4 26B-A4B instruction-tuned checkpoint, text-only, on Apple Silicon Macs (M-series) with macOS 26 and Metal 4. It is not a general-purpose toolkit for any model.
No vision support. The full Gemma 4 26B-A4B model supports image, video, and audio input. TurboFieldfare is text-only — the vision encoder (~550M parameters) is not included.
Prefill latency. Time-to-first-token is slow on lower-end hardware. On the M2 Air, a 121-token prompt took ~8.8 seconds before the first output token. This improves on faster chips, but interactive chat has a noticeable initial latency on base-model Macs.
How to Try It Yourself
If you have an M-series Mac with macOS 26, you can run it in a few commands:
# 1. Clone the repository
git clone https://github.com/drumih/turbo-fieldfare.git
cd turbo-fieldfare
# 2. Build the release
swift build -c release
# 3. Launch the Mac app and download the model on first run
.build/release/TurboFieldfareMac
The first run downloads and repacks the pinned model checkpoint (~15GB transfer) from Hugging Face into the .gturbo format. The installer streams byte ranges directly without staging a full shard on disk, so it does not need 14GB of free temp space.
After the model downloads, the workflow is: load the model → type your prompt → press Cmd+Return to generate. Escape stops generation.
For CLI usage without the GUI:
# Raw completion
swift run -c release TurboFieldfareCLI \
--model scratch/gemma4.gturbo \
--prompt "Explain Mixture-of-Experts in one paragraph." \
--max-new 128 --temperature 0
# Instruction chat (provide a messages JSON file)
swift run -c release TurboFieldfareCLI \
--model scratch/gemma4.gturbo \
--messages-file messages.json
There is also a loopback OpenAI-compatible server for connecting other apps to it locally.
What Does This Mean for Local AI?
If you use AI for your work, run a small business, or build AI tooling, this technique matters for three reasons:
It unlocks large-model quality on low-RAM machines. The cheapest Mac Apple sells — an 8GB MacBook Air — can now run a 26B model that previously required a 16GB+ machine. If your daily driver is a base-config Mac, you are no longer capped at 3B-8B models for local inference.
It reframes the hardware question. When SSD throughput becomes the bottleneck, memory bandwidth and RAM size matter less for MoE models. A future where a mid-tier iPhone or MacBook Air runs a capable MoE model locally is no longer far-fetched — the architectural foundation is here.
It points toward energy and cost trade-offs for production. For a business running inference at scale, streaming experts from SSD trades capital expense (less RAM per server) against operational cost (higher energy per token, more SSD wear). Understanding this spectrum helps with infrastructure planning — and our guide to cutting large AI bills covers the other half of that equation.
For a deeper comparison of how local LLM inference fits into a broader cost strategy — including when to go local versus cloud — see our analysis of budget AI model cost comparisons.
How Does This Fit With Other Local AI Approaches?
SSD expert streaming is one technique in a broader ecosystem of on-device AI. Here is how it fits alongside other approaches we have covered:
Quantization-aware training (QAT): Google's official QAT versions of Gemma 4 compress model weights to fit in as little as 1GB for mobile deployment. Our Gemma 4 QAT local AI guide covers how this works and when it is the better choice (smaller models, fully resident, faster). TurboFieldfare and QAT are complementary — QAT shrinks the whole model; SSD streaming leaves it large but loads only what is active.
Sparse attention at scale: MiniMax M3's sparse attention architecture reduces the memory needed for long-context inference, addressing the KV cache side of the problem that streaming leaves resident. See our guide to serving AI agents at scale with sparse attention.
Tiny-model edge inference: If you need even lower resource use — sub-1GB — there are approaches like running compact models on microcontrollers. Our $8 ESP32-S3 LLM guide shows the extreme low end.
The right tool depends on your constraint: QAT for mobile, streaming for large MoE models on desktop, sparse attention for long context, and tiny models for edge devices.
What This Means for You
If you have a base-model Mac (8GB) that you assumed could not run a serious LLM locally — try TurboFieldfare. It is open-source, genuinely functional, and the technique generalizes to future MoE models. For production or server workloads, weigh the SSD wear and energy costs carefully, but for personal experimentation, document Q&A, and private local inference, the trade is overwhelmingly positive.
The broader lesson for builders: MoE architectures are not just about training efficiency — they create an inference opportunity. The sparsity built into MoE models makes SSD expert streaming possible, and unified memory makes it fast enough to be practical. Expect more tools to exploit this in the coming year.
FAQ
Q: What model does TurboFieldfare run? A: TurboFieldfare runs the instruction-tuned Gemma 4 26B-A4B model from Google. It is a Mixture-of-Experts model with 25.2 billion total parameters, but only ~3.88 billion parameters activate per token (8 of 128 experts plus 1 shared expert). Note that TurboFieldfare is an independent project unaffiliated with Google.
Q: Do I need a Mac with a lot of RAM? A: No. The minimum is an Apple Silicon Mac with 8GB of RAM. The memory footprint during inference is approximately 2GB (weights + KV cache). The full 14.3GB model stays on the SSD and experts are streamed on demand.
Q: Can I run this on a Windows PC or Linux? A: TurboFieldfare itself is Apple-Silicon-only (macOS 26, Metal 4, Swift). The SSD expert-streaming technique has been replicated on Windows with tools like llama.cpp for similar MoE models, but performance is lower because discrete GPUs require a PCIe bus copy from system RAM to VRAM that Apple Silicon's unified memory avoids.
Q: Will streaming from SSD wear out my drive? A: For typical interactive use (a few hundred to a few thousand tokens per session), SSD wear is negligible. For sustained server-grade workloads — tens of thousands of tokens daily — SSD read traffic is higher than a fully-resident model and drive endurance should be monitored. Refer to your SSD's TBW (terabytes written) rating for planning.
Q: Is TurboFieldfare faster than Ollama or MLX?
A: No — if the model fits in RAM. TurboFieldfare's M5 Pro speed is 31–35 tok/s versus 76–82 tok/s for mlx-lm on the same machine. The point is not to be faster; it is to work on hardware where the model cannot fully reside in memory (8GB Macs). If you have the RAM to load the full model, standard tools are faster.
Q: Does TurboFieldfare support images and video? A: Not currently. The full Gemma 4 26B-A4B model is multimodal (text, image, video), but TurboFieldfare's scope is text-only inference. The vision encoder is excluded from the streamed model bundle.
Discussion
0 comments