Verdict: Inkling is a 975-billion-parameter open-weight model from Thinking Machines Lab that is explicitly designed to be a starting point for fine-tuning, not a finished, out-of-the-box assistant. If you have a narrow, high-volume task, real labeled data, and a way to grade output — especially anything involving raw audio — fine-tuning a model like Inkling on Tinker is one of the highest-leverage things you can do right now. But if you need a model to simply run well as-is, or you want it to learn facts it doesn't already know, Inkling is the wrong tool: use a generalist frontier API or a retrieval-augmented system instead.
Last verified: July 24, 2026
- Inkling: 975B total params, 41B active (Mixture-of-Experts), Apache 2.0, weights on Hugging Face.
- Best for: narrow, high-volume tasks where you can label outputs and want audio handling.
- Not for: out-of-the-box general chat, teaching a model new facts it lacks.
- Method: fine-tuning uses LoRA (low-rank adaptation) — you train lightweight adapters, not the full 975B.
- Pricing/limits change often — this was last checked 2026-07-24.
What is Inkling and why is it different from GPT or Claude?
Inkling is a Mixture-of-Experts (MoE) transformer released July 15, 2026 by Thinking Machines Lab — the startup founded by former OpenAI CTO Mira Murati, with co-founders John Schulman and Lilian Weng. The model carries 975 billion total parameters but only activates about 41 billion per token because each layer routes each token to just 6 of 256 available experts. That sparsity is what makes a near-trillion-parameter model affordable enough to actually run and fine-tune. It was pretrained on 45 trillion tokens of text, images, audio, and video, and supports a context window of up to 1 million tokens (Thinking Machines Lab launch post).
What sets Inkling apart is not pure benchmark dominance — Thinking Machines is openly clear about that. The model has solid instruction following (79.8% on IFBench) (BenchLM.ai) and competitive agentic scores (63.8% on Terminal-Bench 2.1) (Thinking Machines launch post), but it trails frontier closed systems like Claude Fable 5 (which scores 84.3% on the same Terminal-Bench row) (BenchLM.ai). The pitch is different: Inkling exists so you can adapt it, rather than rely on it as-is.
The other differentiator is genuinely novel: Inkling feeds audio and images straight in as raw data — audio as discrete dMel spectrograms, images as 40x40 pixel patches — rather than transcribing or captioning them first and feeding the text to the language model. That avoids data loss from compression. It is the only model on the Tinker platform that takes raw audio at all as of release day (Thinking Machines launch post); Hugging Face blog).
When should you fine-tune a model at all — and when should you not?
The rule of thumb: fine-tuning teaches a model how to answer, not what to know. If you need the model to learn facts it doesn't already know, fine-tuning is often the wrong tool — research consistently shows retrieval-augmented generation (RAG) outperforms unsupervised fine-tuning for knowledge injection (Ovadia et al., Fine-Tuning or Retrieval? Comparing Knowledge Injection in LLMs, EMNLP 2024 — ACL Anthology). Fine-tuning is also documented to degrade reasoning in some cases, because it can erode a model's chain-of-thought — and smaller models suffer the most.
Fine-tuning earns its keep when:
- You need a rigid structured output style, tone, or format — where passing thousands of examples via a prompt would be impractical or exceed context.
- You need to perform a narrow, high-volume task where the cost of running a frontier general model per call is much greater than the per-call cost of a fine-tuned smaller model.
- You have real labeled data and a grader (a way to score output quality automatically or with human review) so you can measure whether fine-tuning actually improved the model.
The most-cited real-world example is Harvey, a legal AI startup. They trained a custom model (via OpenAI's Custom Model program) to extract citations from legal documents, and lawyers preferred Harvey's custom model over GPT-4 alone 97% of the time in tests across 10 large law firms (Harvey platform page; mmntm.net deep dive). The lesson generalizes to any high-stakes domain: finance, healthcare, legal — where the cost of being wrong is high enough that "good enough" from a general model is not good enough.
When NOT to fine-tune
- You need the model to know things it doesn't. Use RAG instead. RAG lets the model look up external facts at query time, and you avoid catastrophic forgetting (the model forgetting what it already learned) (Ovadia et al., EMNLP 2024).
- You want better reasoning. Fine-tuning can actually degrade a model's chain-of-thought — smaller models suffer the most.
- You have no way to grade output quality. Without a grader or labeled eval set, you cannot tell whether fine-tuning improved or hurt the model. Do not fine-tune blind.
- Your task is one-time, exploratory, or not high-volume. The setup cost (data labeling, training runs, eval harness) does not pay back.
For a deeper treatment of the RAG-vs-fine-tuning decision (and the hybrid pattern where you do both), we break down the production trade-offs in our guide to the Nemotron 3 Embed + Qwen 3.8 open AI memory stack — which covers when retrieval is the right lever and when adaptation is.
How does fine-tuning Inkling actually work — and what is LoRA?
You are not retraining the full 975-billion-parameter model. Fine-tuning a model that size from scratch would require a GPU cluster most organizations cannot rent, much less own. Instead, Tinker uses LoRA (Low-Rank Adaptation) — a parameter-efficient fine-tuning method first introduced in 2022 by researchers at Microsoft (original LoRA paper, Hu et al., arXiv:2106.09685; ICPE 2025 performance study, LCA@UT Austin).
LoRA in one paragraph
LoRA freezes the original model weights and injects two small, trainable "low-rank" matrices alongside each layer. Only those small adapters are updated during training — the base 975B stays frozen. Because the adapters are tiny compared to the full weight matrices, you need a fraction of the GPU memory and training time of full fine-tuning, and the resulting checkpoint is small. At inference time, the adapters can be merged back into the base weights (for speed) or kept separate (so you can hot-swap between many fine-tuned variants on one base model).
Why LoRA matters for small teams
Full fine-tuning of a 975B-parameter model is impractical for almost anybody outside Alpine-scale labs. LoRA makes it realistic for a research group or a small company to fine-tune a model of this size on managed infrastructure like Tinker, where the platform handles GPU allocation, gradient checkpointing, and fault tolerance. The Tinker Cookbook (open-source on GitHub) ships recipes for supervised fine-tuning, reinforcement learning, and preference optimization — so you start from a working template, not a blank file.
The Inkling self-fine-tuning demo
Thinking Machines' launch demo did something genuinely worth knowing about: they had Inkling fine-tune itself. The model was told to become a "lipogram assistant" that never uses the letter "e," a behavior you cannot reliably achieve with prompting alone. Inkling wrote its own training-job plan, generated synthetic training data, wrote a scoring function (any response containing "e" scores zero), launched the supervised fine-tuning run through the Tinker API, evaluated the checkpoint against its base self, and hot-swapped the new weights into the live session. The full loop completed in about 27 minutes (Thinking Machines launch post; PJFP analysis).
This is not a magic button. It is, however, a signal of where tooling is going: the same model you want to customize can drive the training loop, generate the grading rubric, and produce its own training data. You still need to define the task and inspect results — but the plumbing is becoming something you can prompt for, not hand-build.
What does Tinker support, and how does Inkling compare to the other models on it?
Tinker (released October 2025) is the fine-tuning platform that ships alongside Inkling. It exposes four API primitives — forward_backward, optim_step, sample, and save_state — and handles GPU scheduling, fault tolerance, and checkpoint management on its end (InfoQ coverage; Thinking Machines Tinker page).
As of Inkling's release, Tinker supports these open-weight models:
| Model | Type | Why you'd pick it over Inkling |
|---|---|---|
| Inkling (Thinking Machines) | 975B MoE / 41B active | Multimodal, native raw-audio input, controllable thinking effort (0–1 slider) |
| DeepSeek-V3.1 | MoE | Strong general reasoning at low active-param cost |
| Kimi K2.6 (Moonshot) | MoE | Vision capable; leading open-weight on several benchmarks |
| Nemotron-3-Nano-30B-A3B (NVIDIA) | MoE | Smallest Nemotron; cheapest entry point |
| Nemotron-3-Super-120B-A12B (NVIDIA) | MoE | Mid-tier balance of size and capability |
| Nemotron-3-Ultra-550B-A55B (NVIDIA) | MoE | Strong agentic scores; Inkling matches it on Terminal-Bench at roughly a third of the tokens |
| GPT-OSS-120B (OpenAI) | MoE | Familiar OpenAI-style behavior, fully open |
| GPT-OSS-20B (OpenAI) | MoE | Lightest in the GPT-OSS family; fastest to fine-tune |
| Qwen3.6-35B-A3B and family (Alibaba) | MoE + dense variants | Vision capable; many size options |
| Inkling-Small (276B / 12B active) | MoE | Coming soon; lower latency and cost for the same multimodal stack |
Sources: Thinking Machines Tinker page, Tinker announcement post.
Why pick Inkling specifically?
Two real reasons, both verified against primary sources:
- Audio. Of every model on Tinker, Inkling is the only one that takes audio at all. Others handle vision — Kimi and most Qwen models do images fine — but none take raw sound (Thinking Machines launch post). Thinking Machines shipped three cookbook recipes on launch day: speech recognition, speaking-style classification (how something was said, not just what was said), and a medical example fine-tuned on dictated prescriptions so the model learns drug names most general models would mangle.
- Controllable thinking effort. Most models give you a coarse switch — GPT-OSS exposes low/medium/high, for instance. Inkling gives you a continuous slider from 0 to 1. Sweeping that slider traces a full performance-vs-cost curve; Thinking Machines' headline efficiency claim is that Inkling matches Nemotron 3 Ultra on Terminal-Bench 2.1 at roughly one third of the generated tokens (Thinking Machines launch post). For high-volume deployments that run millions of inferences, that token-cost gap is the whole ballgame.
If your job involves audio, or if your inference budget is tight enough that controllable thinking is a real lever — Inkling is the pick. If not, a smaller or cheaper model on Tinker may be the better base. See our comparison of Gemini 3.6 Flash vs 3.5 Flash-Lite for the same model-selection logic applied to closed Google models.
How to decide whether to fine-tune Inkling (a worked checklist)
Use this checklist to decide before you spend any compute. Each item maps to a concrete risk or prerequisite.
- Is your task narrow and repeatable? A single, well-defined input/output shape — extract a citation, classify a spoken intent, produce a fixed-format report — is the sweet spot. If the task is open-ended chat or general Q&A, do not fine-tune; use a generalist API. (See our guide to running local AI in 2026 for when to host models yourself at all.)
- Do you have at least a few hundred labeled examples? More is better. Quality matters more than quantity — a few hundred clean, representative pairs will usually beat ten thousand noisy ones.
- Do you have a grader? You need a way to score outputs automatically (a rule-based check, a model-as-judge, a ground-truth comparison) or a human reviewer in the loop. If you cannot tell whether the fine-tuned model is better, do not ship it.
- Is the task high-volume? If you will run the model millions of times, the per-call savings from a fine-tuned smaller model can dwarf training cost. If the task runs once a week, skip fine-tuning and prompt-engineer instead.
- Does the model already know the facts, or do you need new facts? If the issue is the model lacks knowledge — new product SKUs, current pricing, fresh case law — use RAG, not fine-tuning. Fine-tuning does not reliably inject new facts (Ovadia et al., EMNLP 2024).
- Does the output need to be consistent and structured? If yes, and prompt engineering cannot get the model there after honest effort — that is exactly the case fine-tuning solves.
If you answered "yes" to items 1, 2, 3, and either 4 or 6 — and "no" to item 5 — you have a real fine-tuning case. Otherwise, stop and revisit RAG or prompt engineering first.
What this means for you
- If you are a small business or founder with a narrow, high-volume AI task (transcription with custom formatting, classification of incoming calls, structured extraction from a known document type), fine-tuning is one of the most underrated levers in 2026. Platforms like Tinker have made it dramatically more accessible than it was even a year ago — you no longer need to own a GPU cluster to do it. The small-business AI agent stack we documented is the kind of place fine-tuning fits once you've outgrown prompt-only automation.
- If you are a researcher or engineer with lab access, the Inkling weights are Apache 2.0 and on Hugging Face. Download them, read the model card, and run the Tinker Cookbook recipes. The self-fine-tuning demo is worth reproducing — it is the clearest signal yet of where model adaptation tooling is going.
- If you are choosing between open and closed models generally, the pattern is now clear: closed frontier APIs win for out-of-the-box breadth; open-weight fine-tuning wins for specialized depth where you control the data and the grader. Inkling leans hard into the second lane. The same dynamic is why our Gemini 3.5 Pro delay writeup recommended a cheaper Flash tier as a stopgap — breadth and freshness are different bets.
- Do not fine-tune to teach new facts. This is the single most common mistake. If the model needs to know something it doesn't, give it RAG or a tool. If the model knows the facts but produces them wrong — that is the fine-tuning case.
FAQ
Q: Is Inkling stronger than GPT-5 or Claude?
A: No, and Thinking Machines says so directly. Inkling is explicitly a "broad, balanced generalist" intended as a base for customization, not a benchmark leader. On Terminal-Bench 2.1 it scores 63.8%, while Claude Fable 5 scores 84.3% and GPT-5.6 Sol scores 91.9% on the same benchmark (BenchLM.ai, July 2026). Inkling's pitch is that you will fine-tune it into something better for your specific job — not that it wins out of the box.
Q: Do I need a huge GPU cluster to fine-tune Inkling?
A: No. Tinker uses LoRA (low-rank adaptation), which freezes the 975B base weights and trains only small adapter matrices. You do not update the full model. Tinker handles GPU allocation on its end. For self-hosting the finished model directly (not via Tinker), the official BF16 checkpoint needs around 2 TB of VRAM and even the NVFP4 version needs about 600 GB — so self-hosting the base model still requires data-center hardware (per the Inkling model card and Gadgets Now coverage, July 2026). The practical path for most teams is fine-tune on Tinker then serve through a hosted provider.
Q: Can I use Inkling for raw audio tasks?
A: Yes — and as of launch day, Inkling is the only model on Tinker that takes audio at all. It ingests audio as discrete dMel spectrograms (frequency buckets) rather than transcribing to text first, which avoids data loss from compression. Thinking Machines shipped three cookbook recipes at launch: speech recognition, speaking-style classification, and a medical example trained on dictated prescriptions (Thinking Machines launch post, July 2026).
Q: Fine-tuning or RAG — which should I use?
A: Use fine-tuning when the model has the right facts but produces them in the wrong format or style, and you can grade the output. Use RAG when the model needs facts it doesn't have (dynamic knowledge, proprietary data, citable answers). The best-supported research (Ovadia et al., EMNLP 2024) shows RAG consistently beats unsupervised fine-tuning for injecting new knowledge, while fine-tuning risks catastrophic forgetting. In practice, the strongest systems combine both: fine-tune for behavior, RAG for knowledge.
Q: What is LoRA, in plain terms?
A: LoRA (Low-Rank Adaptation) freezes a pretrained model's weights and adds two small trainable matrices to each layer. Only those tiny adapters are updated during fine-tuning, so you need a fraction of the compute and memory of full fine-tuning. The resulting adapter is small enough to hot-swap between many fine-tuned variants of the same base model. It was introduced by Hu et al. in 2022 and is now the standard parameter-efficient fine-tuning method for large language models (arXiv:2106.09685; ICPE 2025 performance study).
Q: How much does Tinker cost?
A: Tinker is usage-based, with published pricing starting at $0.09 per million tokens (EveryDev.ai, July 2026). Thinking Machines also offered a limited-time 50% discount on Inkling fine-tuning at launch with no public end date (IntelligentHQ, July 2026). Pricing and limits are volatile — re-check the official Tinker docs before budgeting.
Discussion
0 comments