Verdict: Bad Theory Labs shipped two models that together solve the hardest problem in local AI — a 35B reasoning model that can plan a complex task, and a 2.7B on-device agent that can execute it without sending a byte to the cloud. BTL-4 scores 78.4% on SWE-bench Verified and 73.5% on BFCL v4 for tool calling, while Macaw runs on any Apple Silicon Mac at 1.21 seconds per action with 97 built-in macOS tools. Both are free to download today under Apache-2.0 and MIT licenses, respectively, but you need different hardware for each and the integration between them is still manual.
What is BTL-4 and what problem does it solve?
BTL-4 is a 35-billion-parameter open-weight reasoning model from Bad Theory Labs, fine-tuned from the Ornith-1.0-35B base (itself a Qwen 3.5 Mixture-of-Experts architecture) on an execution-gated reasoning corpus — meaning the training data was filtered to keep only reasoning traces where the resulting code actually ran and passed its tests (HuggingFace model card). It is built specifically for tool use, software engineering, and long-horizon agent work, not for chat.
The key numbers: it scores 78.4% on SWE-bench Verified (the human-verified subset of real GitHub issue resolution from repos like Django, Flask, and scikit-learn), 73.5% on BFCL v4 (the Berkeley Function-Calling Leaderboard that measures whether a model can correctly identify and fill parameters for tool calls — the metric that matters most for agent builders), and 66.1% on LiveCodeBench v6 (HuggingFace model card). For context, the top of the SWE-bench leaderboard is Claude Opus 5 at 96%, and the best open-weight models like DeepSeek V4 Pro Max hover around 80.6% — BTL-4 at 78.4% is in the same tier as many proprietary frontier models and is within 2.2 points of DeepSeek (BenchLM leaderboard).
The architecture is a Mixture of Experts with approximately 2.1 billion active parameters per token out of 35 billion total, which means it uses a large model's memory footprint but only a small model's compute per inference step — making it surprisingly fast for its size. It has a 262,144-token context window.
What is Macaw and how does it work on a Mac?
Macaw is a 2.7-billion-parameter on-device AI agent that runs entirely on macOS and controls your machine through 97 verified tools covering mail, files, calendar, music, system settings, and multi-step chains (GitHub README). It is a fine-tune of Liquid AI's LFM2.5-2.6B, a hybrid model designed for edge deployment with multiplicative gates and short convolutions (Liquid AI docs). The 4-bit MLX build is approximately 1.5 GB on disk.
On a base Apple M2, Macaw achieves 10/10 tool-call accuracy with a mean request time of 1.21 seconds and decode speed of 40.3 tokens per second (GitHub README). It runs as a Swift menu-bar app with a floating prompt bar (Option + Space), backed by a local mlx_lm server on Metal. No cloud, no API key, no network connection required.
The safety design is notable: only a tool name plus literal arguments cross from the model to the machine, validated against a tools.json schema. The model never writes the AppleScript that runs — the app does. This means untrusted text the model reads (mail subjects, filenames, web pages) cannot talk it into executing arbitrary script (GitHub README).
How do BTL-4 and Macaw work together as an agent stack?
The pairing is brain-plus-hands: BTL-4 reasons, plans, and produces structured outputs (scripts, briefs, code, content); Macaw executes on your machine (moves files, opens apps, reads documents, chains macOS tools). Neither model talks to the other directly — you orchestrate the handoff yourself by feeding BTL-4's output into a Macaw prompt, or by scripting the pipeline with a shell script or Python glue.
This is the open-source world's first practical attempt at a local agent stack where one model thinks through a multi-step task and another carries it out. The idea is that BTL-4's 262K context window lets it hold an entire project brief plus conversation history while it plans, and Macaw's 97 tools let it act without you touching Finder, System Settings, or Terminal.
Here is what a combined workflow looks like in practice:
- Feed BTL-4 your raw context (community questions, project notes, content backlog) — its 262K context window holds it all without truncation.
- Ask BTL-4 to analyze, sort, and produce structured deliverables (scripts, briefs, file names, content plans).
- Hand the output to Macaw with an execution prompt: "Move these files, rename them, archive last week's materials, open this app."
- Macaw executes each action at roughly 1.2 seconds per tool call, entirely locally.
The bottleneck today is the integration layer — you build the glue. But both models expose OpenAI-compatible APIs (BTL-4 via vLLM or llama.cpp, Macaw via its built-in mlx_lm server at localhost:8138), so a Python script can call both endpoints and chain them.
How much does it cost to run BTL-4 and Macaw?
Both models are free to download. The total cost is hardware and electricity.
| Model | License | Download size | Minimum hardware | Cost |
|---|---|---|---|---|
| BTL-4 (BF16) | Apache-2.0 | ~70 GB | 24 GB+ VRAM GPU or 64 GB+ unified memory | Free |
| BTL-4 (GGUF quantized) | Apache-2.0 | ~10–25 GB (varies by quant) | 16–32 GB RAM | Free |
| Macaw (4-bit MLX) | MIT (+ LFM Open License) | ~1.5 GB | Apple Silicon Mac, 8 GB+ RAM | Free |
| Macaw (BF16) | MIT (+ LFM Open License) | ~5 GB | Apple Silicon Mac, 16 GB+ RAM | Free |
Sources: BTL-4 on HuggingFace, Macaw on GitHub
One important licensing note: Macaw's weights are derivatives of LFM2.5-2.6B under the LFM Open License v1.0, which carries a commercial restriction — entities with $10 million or more in annual revenue are restricted. The code itself is MIT with no such limit. BTL-4 is Apache-2.0 with no revenue-based restriction.
What hardware do you need to run the full stack?
Running both models requires two different hardware profiles. BTL-4 is a large model that needs substantial GPU or unified memory, while Macaw is lightweight but only runs on Apple Silicon.
For BTL-4:
- A single GPU with 24 GB+ VRAM (for Q5_K_M quantization at ~25 GB) — an NVIDIA RTX 3090/4090 or A5000-class card
- Or an Apple Silicon Mac with 64 GB+ unified memory (the BF16 model requires ~70 GB VRAM per LLM Explorer)
- Or a cloud GPU instance (H100, A100) at approximately $2–4/hour on-demand if you do not have local hardware
- The MoE architecture means only ~2.1B parameters activate per token, so inference is faster than a dense 35B model
For Macaw:
- Any Apple Silicon Mac (M1, M2, M3, M4) with 8 GB+ RAM
- macOS 14+ (Sonoma or later)
- Screen Recording permission for screen reading, Accessibility permission for app control
If you already have an Apple Silicon Mac with 32 GB+ unified memory, you can run both models on the same machine. BTL-4 will run via llama.cpp (GGUF) or vLLM, and Macaw runs natively via MLX on Metal.
How do you set up BTL-4 locally?
The model card provides official serving instructions for four runtimes. Here is the fastest path for each:
Option 1: vLLM (GPU server, recommended for tool use)
vllm serve badtheorylabs/BTL-4 \
--max-model-len 131072 \
--enable-auto-tool-choice --tool-call-parser qwen3_xml \
--reasoning-parser qwen3 \
--trust-remote-code
Option 2: llama.cpp (local GGUF, runs on CPU or GPU)
llama-server -m BTL-4-IQ2_XXS.gguf --port 8080 \
--jinja \
--reasoning-format deepseek \
-c 32768 -fa on \
--cache-type-k q8_0 --cache-type-v q8_0
Option 3: SGLang (GPU, alternative to vLLM)
pip install sglang
python3 -m sglang.launch_server \
--model-path "badtheorylabs/BTL-4" \
--host 0.0.0.0 --port 30000
Option 4: Docker Model Runner
docker model run hf.co/badtheorylabs/BTL-4
A critical configuration note from the model card: reasoning must be separated from content on every stack. With vLLM, use --reasoning-parser qwen3; with llama.cpp, use --reasoning-format deepseek. Without it, reasoning accumulates into content each turn and the model repeats turns instead of terminating. The recommended generation settings are temperature 1.0, top_p 0.95.
The model card also reveals an important finding: LiveCodeBench improved from 60.9% to 66.1% purely by raising the output budget from 16K to 32K tokens. At 16K, 23.5% of problems were truncated mid-solution. The takeaway is to give BTL-4 room to reason — set max_new_tokens to 32K or higher for hard tasks (HuggingFace model card).
How do you set up Macaw locally?
Macaw's setup is simpler since it targets only macOS with a native app:
Step 1: Serve the model
pip install mlx mlx-lm
python -m mlx_lm.server --model badtheorylabs/Macaw-4bit-MLX --port 8138
Step 2: Build and run the macOS app
git clone https://github.com/Badtheorylabs/Macaw.git
cd Macaw/app
swift build
The app runs in the menu bar, starts the server automatically, and exposes a floating prompt bar via Option + Space.
Step 3: Install the runtime helpers
mkdir -p ~/.macaw
cp runtime/*.py runtime/tools.json runtime/tools_prompt.json ~/.macaw/
Step 4: Grant macOS permissions
- System Settings → Privacy & Security → Screen Recording → enable for Macaw
- System Settings → Privacy & Security → Accessibility → enable for Macaw
Source: GitHub README
How do you chain BTL-4 and Macaw for real tasks?
The key workflow pattern is: send context to BTL-4 for analysis/planning, then send the structured output to Macaw for execution. Here is a concrete three-step template you can adapt:
Step 1 — Analyze with BTL-4 Send your raw material (community threads, meeting notes, project briefs) to BTL-4 with a prompt like: "Here are 18 member questions from this week's community thread. Group them into themes. Draft a brief for each theme with bullet-point answers. Keep it practical."
BTL-4's 262K context window means you can paste an entire week of conversations without truncation. It holds the full thread, reasons across it, and produces structured output.
Step 2 — Hand off to Macaw Take BTL-4's output and give Macaw an execution prompt: "Move the new briefs into this week's content calendar folder, rename them with today's date, archive last week's materials, and open the slide template."
Macaw executes each action at approximately 1.2 seconds per tool call. It does not write scripts — it calls verified tools from its 97-tool library, validated against the tools.json schema.
Step 3 — Automate with glue
Write a Python script that calls BTL-4's API (vLLM or llama.cpp) and Macaw's API (localhost:8138) to chain the two automatically. Both expose OpenAI-compatible /v1/chat/completions endpoints, so you can use the openai Python library pointed at two different base URLs.
This is where the stack sits today — powerful but requiring technical comfort. The models are excellent individually, and the pairing is sound in principle, but the orchestration is up to you.
BTL-4 and Macaw compared to alternatives
| Feature | BTL-4 35B | Macaw 2.7B | Cloud agents (Claude, GPT) |
|---|---|---|---|
| Where it runs | Your hardware (GPU/Mac) | Your Mac (Apple Silicon) | Vendor cloud |
| Data leaves your machine | Never | Never | Yes |
| Cost per token | $0 (electricity) | $0 (electricity) | $3–15 per M tokens |
| Model size | 35B (2.1B active) | 2.7B | Proprietary |
| SWE-bench Verified | 78.4% | Not rated | Claude Opus 5: 96% |
| BFCL v4 (tool calling) | 73.5% | 10/10 on own bench | Varies |
| Context window | 262K | 128K (from base LFM2.5) | 200K–2M |
| License | Apache-2.0 | MIT + LFM Open (commercial cap at $10M revenue) | Proprietary |
| Setup difficulty | Moderate (GPU/serving) | Low (Swift app + pip) | None (sign up) |
Sources: BTL-4 model card, Macaw GitHub, BenchLM, Liquid AI docs
What does this mean for you?
For small business owners: This stack gives you the building blocks of an AI automation pipeline without monthly API fees — BTL-4 can draft content, analyze data, and plan workflows, while Macaw can handle file management and app control on your Mac. The tradeoff is setup effort and the need for capable hardware. If you do not have an Apple Silicon Mac, you can run BTL-4 on a cloud GPU and separate Macaw from the equation.
For developers and builders: BTL-4's 78.4% SWE-bench score puts it in the frontier open-weight tier, and the execution-gated training approach is the differentiator — it was trained on reasoning that actually led somewhere. The 73.5% BFCL v4 score means it is genuinely good at tool calling, not just coding. If you are building agent infrastructure and want open-weights, this is one of the strongest options near this parameter count. Pair it with Macaw if your agent needs to act on macOS, or use it standalone with your own agent framework.
For privacy-sensitive teams: Both models run entirely offline with no network dependency. No API keys, no accounts, no telemetry. If your use case involves proprietary data or regulated workloads, this stack keeps everything on-device.
Is the BTL-4 + Macaw stack ready for production use?
For individual developers and small teams willing to handle deployment, yes — both models are live today (not waitlists), with published benchmarks and working code. BTL-4 is available on HuggingFace with weights, model card, and serving instructions for four runtimes. Macaw is on GitHub with a Swift app, Python runtime, and 97 verified tools. The pieces are real, not hype.
The gap is in the integration layer. There is no official "BTL-4 + Macaw" product that chains them automatically. You build the glue. The models are also still early in their lifecycle — BTL-4 had 571 downloads in its first month, and Macaw has 24 stars on GitHub as of this writing. Community tooling, quantizations, and integrations will mature over time.
For a deeper look at how open-weight models fit into the broader AI agent landscape, see our open-source AI agent tools roundup and our guide on when to trust and optimize your own open-source models.
Limitations and honest caveats
BTL-4 is not a Claude replacement. Its 78.4% on SWE-bench is strong for a 35B open-weight model, but Claude Opus 5 scores 96% and GPT-5.5 scores 82.6% on the same benchmark (BenchLM). For the hardest coding tasks, proprietary frontier models still lead, though BTL-4 closes the gap significantly for the parameter count and cost (free vs. per-token).
Macaw only runs on Apple Silicon. If you are on Linux or Windows, Macaw does not work — it needs macOS 14+, the MLX framework, and Apple's Metal GPU API. There is no cross-platform build.
Macaw's commercial license is restricted. The MIT code is free for everyone, but the model weights are derivatives of LFM2.5-2.6B under the LFM Open License, which restricts commercial use by entities with $10 million or more in annual revenue. BTL-4's Apache-2.0 license has no such limit.
BTL-4 requires real GPU hardware. The BF16 model needs roughly 70 GB of VRAM. You can run the GGUF quantized version on 16–32 GB of RAM, but it will be slower. On Apple Silicon, you need 64 GB+ unified memory for the full-precision model.
Benchmarks are vendor-reported. BTL-4's scores were run in-house with official scorers. The BFCL comparison was a paired test against the base Ornith model using identical harness and decoding. While the methodology is sound, these are not independent third-party reproductions.
Q: Can BTL-4 and Macaw talk to each other directly? A: No. There is no built-in integration between the two models. You orchestrate the handoff yourself by calling BTL-4's API for reasoning and Macaw's API (localhost:8138) for execution. Both expose OpenAI-compatible endpoints, so Python glue is straightforward.
Q: Does BTL-4 work on a regular laptop without a GPU? A: Yes, but slowly. The GGUF quantized version runs on CPU via llama.cpp. With the IQ2_XXS quantization, the model fits in roughly 10 GB of RAM, but generation will be measured in seconds per token rather than tokens per second. For practical use, you want a GPU with 24 GB+ VRAM or an Apple Silicon Mac with 32 GB+ unified memory.
Q: What is BFCL v4 and why does the 73.5% score matter? A: BFCL v4 is the Berkeley Function-Calling Leaderboard, a benchmark that tests whether a model can correctly identify which function to call and fill its parameters with valid values. It covers agentic multi-step tasks, multi-turn conversations, live API calls, and static test cases. The 73.5% score matters because tool-calling accuracy is the single biggest bottleneck in production AI agents — a model that produces malformed JSON or picks the wrong tool will break your pipeline regardless of its reasoning ability.
Q: Is Macaw better than Siri or Apple Intelligence for Mac automation? A: Macaw is designed for a different use case than Siri. Siri handles voice commands and simple actions; Macaw chains 97 specific macOS tools across mail, files, calendar, music, and system settings in plain English, with 10/10 tool-call accuracy. Macaw runs entirely locally with no cloud dependency, while Siri and Apple Intelligence route through Apple's servers. Macaw is better for complex multi-step automation; Siri is better for quick hands-free commands.
Q: What is the difference between BTL-4 and Ornith-1.0-35B? A: Ornith-1.0-35B is the base model — a 35B Mixture-of-Experts coding model from DeepReinforce, built on the Qwen 3.5 MoE architecture. BTL-4 is fine-tuned from Ornith-1.0-35B specifically on execution-gated reasoning traces, meaning the training data was filtered to keep only examples where the resulting code actually ran and passed tests. BTL-4 scores higher on benchmarks: 73.5% on BFCL v4 vs Ornith's 69.2%, and 78.4% on SWE-bench Verified.
Q: Can I use these models for commercial purposes? A: BTL-4 is Apache-2.0 licensed with no commercial restrictions. Macaw's code is MIT, but its model weights are derivatives of Liquid AI's LFM2.5-2.6B under the LFM Open License v1.0, which restricts commercial use by entities with $10 million or more in annual revenue. Check the LFM Open License before deploying Macaw in a commercial product above that revenue threshold.
Q: How much RAM do I need to run the full BTL-4 + Macaw stack on a Mac? A: For the GGUF quantized BTL-4, you need 16–32 GB of unified memory depending on the quantization level. For the BF16 version, you need roughly 70 GB. Macaw itself only needs about 2 GB of RAM for the 4-bit MLX build. To run both simultaneously on one Mac, 32 GB+ unified memory is the practical floor.
Every claim here is traced to a primary source, dated, and listed under Sources. Research and drafting are AI-assisted; editing, verification and publication are human decisions, and a person is accountable for what appears on this page. How we work →
Discussion
0 comments