Verdict: Data curation — not compute, not model architecture — is the single highest-leverage lever in LLM post-training today. Frontier labs and open-source research teams alike have independently converged on the same finding: a well-curated 114,000-example dataset can outperform a poorly-curated million-example one, and the right recipe scales predictably. The OpenThoughts project proved this with empirical scaling laws across four dataset generations, reaching 53% on AIME 2025 with a 7B model trained on publicly available data alone.
Last verified: 2026-08-01
- Data quality beats data quantity: a curated 114K dataset matched or beat larger but noisier ones.
- The proven pipeline: source questions → mix → filter questions → generate answers with teacher models → filter answers → deduplicate.
- Sampling multiple answers per question (up to 16x) improves results — counterintuitively, diversity beats volume of questions.
- Stronger teacher models are NOT always better teachers; the best teacher depends on the task.
- Open-source tools (Bespoke Curator, OpenThoughts datasets) make this pipeline reproducible today.
What Is Data Curation for Post-Training LLMs?
Data curation for post-training is the systematic process of selecting, filtering, generating, and balancing training data so that supervised fine-tuning (SFT) or reinforcement learning (RL) actually moves the metrics you care about. It treats data quality as a first-class engineering problem — not something you hope to get right, but something you measure, ablate, and scale.
Post-training is everything that happens after a foundation model finishes pre-training. It includes SFT (teaching a model to follow instructions or reason step-by-step), preference optimization (RLHF, DPO), and reinforcement learning from environment feedback (RL where the model interacts with tools or sandboxes). In all three cases, the model already has the raw capability from pre-training — post-training is what shapes it into something useful, reliable, and aligned.
The bottleneck shifted from compute to data for a clear reason. Pre-training infrastructure is now commoditized: providers like Fireworks, Tinker, and Together AI make managed fine-tuning accessible, and open-weight models from DeepSeek, Qwen, and Meta are freely available. What enterprises and research teams consistently lack is access to the high-quality, well-shaped data that actually drives post-training improvement.
Why Is Data the Bottleneck in Post-Training?
The gap between a pre-trained model and a deployed one is entirely a data problem. Pre-training gives a model broad knowledge from trillions of web tokens. Post-training is where domain expertise, reasoning style, tool use, and behavioral alignment get encoded. That encoding only works if the training data is clean, diverse, and well-structured.
Consider what happens with poor data: a model fine-tuned on noisy or duplicated examples learns to reproduce errors. A model trained on math problems with incorrect answer traces learns to reason badly with confidence. A model trained on too-easy questions never stretches its reasoning depth. These are not theoretical risks — the OpenThoughts team documented exactly these failures across 1,000+ controlled experiments, then fixed them systematically to build their scaling law.
The enterprise version of this problem is even starker. Companies have proprietary logs, internal documentation, and domain-specific workflows, but that raw data is unstructured, inconsistent, and rarely in the question-answer format that post-training needs. The entire field of "data curation" exists to bridge that gap — turning messy, real-world text into the kind of structured, verified training data that reliably improves a model.
What Does a Proven Post-Training Data Pipeline Look Like?
The OpenThoughts project, a collaboration between Bespoke Labs, Stanford, UC Berkeley, University of Washington, and other institutions, distilled a six-stage pipeline that demonstrably scales. Published as an ICLR 2026 oral paper, it represents the most systematically studied open-source data curation recipe for reasoning models to date.
The Six-Stage Pipeline
Here are the stages, abstracted from the OpenThoughts paper (arXiv:2506.04178) and the Bespoke Curator tool:
| Stage | What Happens | Key Knobs |
|---|---|---|
| 1. Question Sourcing | Pull prompts from existing datasets (math competitions, code platforms, science Q&A) or generate synthetic questions | Source selection, domain balance |
| 2. Question Mixing | Decide the ratio of questions from each source to create a balanced dataset | Domain proportions, difficulty distribution |
| 3. Question Filtering | Remove low-quality, too-easy, or duplicated questions using fastText classifiers or LLM judges | Filter model choice, hardness thresholds |
| 4. Answer Generation | Use a teacher model to produce reasoning traces for each question | Teacher model selection, number of samples per question |
| 5. Answer Filtering | Verify generated answers — LLM judges, majority consensus, ground-truth comparison | Verification method, consensus threshold |
| 6. Deduplication | Remove duplicate questions and redundant answer patterns to maximize data diversity | Similarity threshold, near-duplicate detection |
The pipeline is not linear — it is iterative. Each stage has knobs that must be tested via ablation (controlled experiments varying one thing at a time), and the OpenThoughts team ran over 1,000 such experiments to find the recipe that scaled.
How Do You Source Questions for Reasoning Data?
Question sourcing is the foundation. You cannot fix a bad question with a better answer — a poorly-specified prompt produces a misleading training signal regardless of how good the teacher model is.
The most effective approach is to source from domains with verifiable answers. The OpenThoughts pipeline draws from math (competition problems from AoPS, AMC, AIME), code (CodeForces, Stack Overflow, programming challenges), and science (graduate-level Q&A). These domains have a critical property: correctness is deterministic. A math problem has a right answer. A coding challenge has passing tests. This means the answer-filtering stage can verify quality objectively.
For enterprise applications, the equivalent is identifying your domain's "verifiable" subset. If you are training a model for customer support, the verifiable signals might be ticket resolution rates or CSAT scores. For a code-generation model, it is test suite pass rates. For a financial analysis model, it is factual accuracy against regulatory filings. The principle is the same: train on data where you can check the answer.
Why Does Sampling Multiple Answers Per Question Work Better Than Using More Questions?
This was one of the most counterintuitive findings from the OpenThoughts research. Generating 16 answers for a single question and then keeping the best one outperforms using 16 separate questions each answered once — even though the total token cost is similar.
The explanation is diversity in reasoning traces. When a student model fine-tunes on multiple reasoning paths to the same answer, it learns that there are many valid ways to reach the correct conclusion. This generalizes better than seeing the same number of answers to different questions, where the model has fewer "perspectives" per problem to learn from.
For practitioners building reasoning data, this means:
- Prioritize depth over breadth when token budget is constrained — answer each question multiple times.
- Use answer diversity as a signal: if 16 attempts produce the same answer with very different reasoning, those traces are high-value training data.
- If 16 attempts disagree, use majority consensus or an LLM judge to select the best trace(s) and discard the rest.
This finding directly informed the OpenThoughts3-1.2M recipe, which scaled to 1.2 million examples by combining question diversification with multi-answer sampling.
Are Stronger Teacher Models Always Better Teachers?
No — and this is the second most important counterintuitive finding. The instinct is to use the most powerful model available (GPT-class or Claude-class) as the teacher for generating reasoning traces. The research shows this is frequently wrong.
In the OpenThoughts experiments, Qwen-based models sometimes produced better training data than Claude-based models for specific domains, despite Claude being the "stronger" model on benchmarks. The Bespoke-Stratos-17k dataset used DeepSeek-R1 as its annotator and achieved strong results, while later OpenThoughts generations found QwQ-32B to be the best teacher for their final recipe.
The practical implication: do not default to the most expensive model. Instead, run small ablations comparing 2-3 teacher models across different domains before committing to one. The best teacher is the one whose training data improves your student model the most — which is not always the one with the highest benchmark score.
| Teacher Strategy | When It Works | Risk |
|---|---|---|
| Largest frontier model (Claude, GPT) | High-stakes, diverse domains where reasoning style diversity matters most | Expensive; may produce over-verbose or overly cautious traces |
| Strong open-weight model (DeepSeek-R1, QwQ-32B) | When you need scalable in-house generation; OpenThoughts uses QwQ-32B | Needs inference infrastructure |
| Mid-size model fine-tuned on your domain | When you have domain-specific data and need consistency | Upfront cost; lower ceiling |
How Do You Filter Low-Quality Answers?
Answer filtering is what keeps bad training signal out of your model. There are three primary approaches, ranked by cost and effectiveness:
1. Ground-truth verification (cheapest and most reliable when available): For math and code problems, compare the final answer to the known correct answer. This is deterministic and cheap. The OpenThoughts pipeline uses this for all domains where ground truth exists — math answers, code test results, and puzzle solutions.
2. LLM judge (moderate cost, broad applicability): Use a second model to evaluate whether an answer is correct and well-reasoned. The OpenThoughts paper used GPT-4o-mini for this stage in early versions. The risk is that the judge model has its own error rate, so it should be calibrated against ground truth in at least a sample of cases.
3. Majority consensus (cheap, works with multi-sampling): If you generate 16 answers per question, the majority answer is more likely correct than any single attempt. Selecting traces that agree with the majority and discarding outliers is a form of automated quality filtering. This works best when the base rate of correct answers is above 50%.
In practice, you combine these: ground truth where available, LLM judge for harder-to-verify domains (science, open-ended reasoning), and majority consensus as a cheap additional filter on top of multi-sampled answers.
What Tools Can You Use for Data Curation Today?
The open-source ecosystem has matured significantly. You do not need to build a data curation pipeline from scratch.
Bespoke Curator
Bespoke Curator (GitHub: bespokelabsai/curator, Apache-2.0 license) is a Python library for generating and curating synthetic data at scale. It was the tool used to create the Bespoke-Stratos-17k and OpenThoughts datasets. Key features:
- Python-based API for defining data generation pipelines
- Built-in caching, async batching, and fault recovery (handles thousands of requests per batch)
- LLM inference via provider integrations (OpenAI, DeepSeek, Anthropic, vLLM, and others)
- Structured output support (JSON schemas)
- Fine-tuning integration: Curator now connects directly to Tinker and Fireworks AI for managed LoRA fine-tuning, so you can go from curated data to a deployed model in a single pipeline
Install with: pip install bespokelabs-curator
OpenThoughts Datasets
OpenThoughts datasets are available on Hugging Face. These are pre-curated reasoning datasets you can use directly for SFT or as a starting point for your own curation:
| Dataset | Size | Teacher Model | Use Case |
|---|---|---|---|
| Bespoke-Stratos-17k | 17K | DeepSeek-R1 | Initial reasoning distillation experiments |
| OpenThoughts-114k | 114K | DeepSeek-R1 | Scaled reasoning SFT, #1 trending on HuggingFace |
| OpenThoughts2-1M | 1M | DeepSeek-R1 | Large-scale reasoning SFT |
| OpenThoughts3-1.2M | 1.2M | QwQ-32B | Best current open-source reasoning recipe |
Terminal-Bench for Agent Training
Terminal-Bench (tbench.ai) is a benchmark for evaluating agents in real terminal environments, developed through a Stanford x Laude collaboration. Terminal-Bench 2.0 includes 89 challenging tasks with human-written solutions and verification tests. Frontier models score below 65% on the benchmark, making it a meaningful evaluation signal for RL-based agent training.
Terminal-Bench is critical for the agent-training side of post-training. Where reasoning data teaches models to think better, RL environments teach agents to act better. Terminal-Bench provides both the task suite and execution harness for measuring whether your agent training is actually improving agent reliability.
| Benchmark | Tasks | Domain | Use Case |
|---|---|---|---|
| Terminal-Bench 2.0 | 89 | Terminal/CLI tasks | Agent RL evaluation |
| SWE-bench Verified | 500 | GitHub issue resolution | Coding agent evaluation |
| OpenThoughts3 | 1.2M examples | Math, code, science | Reasoning model SFT |
How Do RL Environments Differ from SFT Data for Post-Training?
SFT data is static — question-answer pairs that a model learns to reproduce. RL environments are dynamic — the model takes actions, receives feedback (rewards), and learns from the consequences. Both are forms of post-training data, but they differ in shape, cost, and infrastructure.
SFT data looks like: {question, reasoning_trace, final_answer}. You generate it once, filter it, and use it to fine-tune via standard language modeling loss. The pipeline is well-understood and the OpenThoughts recipe is a complete reference.
RL environments look like: {task_definition, sandbox, reward_function, toolset}. The model must interact with the environment, take actions (run commands, write code, query APIs), and the reward function evaluates whether the agent achieved the goal. This requires sandboxed compute (Docker, gVisor, or cloud sandboxes), checkpointing for long-horizon rollouts, and orchestration infrastructure to manage many parallel agent sessions.
The "reference stack" for RL environments includes:
- Environment layer: How you build and version RL environments — defining tasks, tool interfaces, and reward functions.
- Sandbox/compute layer: Sandboxed execution environments for rollouts, with snapshotting and rollback for long-horizon tasks.
- Training layer: SFT, RL (GRPO, PPO), and prompt optimization (like GePA — Generative Prompt Adaptation, which uses LLMs to optimize system prompts based on reflection).
Post-training for agents is newer and less mature than SFT for reasoning. The OpenThoughts-Agent project is the leading open-source effort here, applying the same data-curation methodology (sourcing, filtering, teacher selection, scaling) to agent trajectories rather than reasoning traces.
What Does This Mean for You?
If you are a developer or small team building with LLMs: You almost certainly do not need to pre-train. Open-weight models from DeepSeek, Qwen, and Meta are excellent starting points. Your competitive advantage is in post-training — and within post-training, your advantage is data curation. Use Bespoke Curator to build a pipeline: source questions from your domain, generate answers with a teacher model, filter aggressively, and fine-tune via LoRA through Tinker or Fireworks.
If you are an enterprise team: The bottleneck is not infrastructure (you can rent it) or models (they are open). It is proprietary data curation. Your logs, customer interactions, and domain knowledge are the raw material. The OpenThoughts pipeline gives you the recipe: source, mix, filter, generate, filter again, deduplicate. Apply it to your domain with your own verifiable signals (CSAT, test pass rates, compliance checks) as the ground truth.
If you are training agents, not just models: RL environments are your data. Terminal-Bench and SWE-bench give you reproducible evaluation. The frontier is reliability — making agents autonomous for longer durations without failure — and post-training on RL environments (not just prompting) is the most powerful lever for improvement.
Related reading
FAQ
Q: What is the difference between pre-training data and post-training data? A: Pre-training data is broad, web-scale text used to build a model's foundational knowledge (trillions of tokens). Post-training data is structured, task-specific, and much smaller (thousands to millions of examples). Pre-training teaches a model what it knows; post-training teaches it how to use that knowledge — how to reason, follow instructions, use tools, and align with human preferences.
Q: Can I use synthetic data for post-training without verified ground truth? A: You can, but the quality is significantly lower and harder to verify. The OpenThoughts pipeline relies heavily on verifiable answers (math solutions, code tests). Without ground truth, your only filtering options are LLM judges and majority consensus, which have their own error rates. For best results, design your training data so that at least a portion has verifiable correctness signals.
Q: How much data do I need for effective post-training? A: Quality matters far more than quantity. OpenThoughts-114k with 114,000 well-curated examples outperformed larger but less curated datasets. Bespoke-Stratos-17k proved that even 17,000 high-quality examples can meaningfully improve a 7B model. Start small, verify your pipeline scales, then increase dataset size. The OpenThoughts paper demonstrated a consistent scaling law — if your recipe is right, more data reliably improves results.
Q: What is a teacher model in post-training? A: A teacher model is a model used to generate the training data (reasoning traces, answers) that a student model fine-tunes on. The teacher does not need to be the most powerful model available — the OpenThoughts project found that Qwen-based models sometimes outperformed Claude as teachers for specific domains. The best teacher is determined empirically, not by benchmark scores.
Q: Is Bespoke Curator free to use?
A: Yes. Bespoke Curator is open-source under the Apache-2.0 license and available on PyPI (pip install bespokelabs-curator). You pay for the LLM inference costs of whatever provider you configure (OpenAI, DeepSeek, Anthropic, etc.) but the tool itself is free. The hosted Curator Viewer (for visualizing data during generation) is also free, with optional paid authentication for private datasets.
Q: What is the relationship between data curation and reinforcement learning for agents? A: Data curation applies to both SFT (reasoning data) and RL (agent environments). For SFT, you curate question-answer pairs. For RL, you curate the environment itself — task definitions, tool interfaces, reward functions, and sandbox configurations. The same principles (source diverse tasks, filter for quality, scale with ablation) apply, but the infrastructure for RL environments is more complex because it requires sandboxed compute and real-time interaction.
Discussion
0 comments