You will never read every line of code an AI agent writes in your codebase again — and pretending otherwise is the most expensive engineering mistake of 2026. The fix is not more code review. The fix is building layered defense systems that make unread code trustworthy: architecture contracts the agent can't violate, design docs that replace review, dependency graphs that flag architectural drift, automated eval pipelines that A/B test code quality, and type systems that lie never. This is the 7-layer Slop Defense Framework — built from first principles, not governance policies.
Last verified: 2026-07-31 — 42% of committed code is AI-generated; 96% of developers don't fully trust AI code; 61% say it "looks correct but isn't reliable" (Sonar 2026 State of Code Survey). Code churn has doubled since pre-AI baselines (GitClear). The framework below is how you keep shipping without drowning.
What is "AI code slop" and why is it getting worse?
AI code slop is any code in your repository that no human has read, understood, and verified — code that compiles and appears to work but harbors hidden edge-case failures, security vulnerabilities, architectural violations, and technical debt that surfaces months later. Andrej Karpathy coined the term "vibe coding" in February 2025 to describe this: "There's a new kind of coding I call 'vibe coding,' where you fully give in to the vibes, embrace exponentials, and forget that the code even exists" (Wikipedia: Vibe coding). It became Collins Dictionary's Word of the Year 2025.
The scale is now undeniable. Sonar's 2026 State of Code Developer Survey — conducted October 2025 among 1,149 professional developers — found that 42% of committed code is now AI-generated or AI-assisted, and developers predict this will reach 65% by 2027 (Sonar). Yet 96% of developers do not fully trust AI-generated code and only 48% always verify it before committing.
The result is a verification bottleneck: code generation has accelerated, but review capacity has not. Meanwhile, GitClear's analysis of 211 million lines of code from 2021 to 2025 found that "moved" code — the refactoring and reuse operations that keep a codebase healthy — plummeted from 25% of all changes in 2021 to less than 10% in 2025 (GitClear Research). Code churn (lines reverted or updated within two weeks) rose from 3.3% in 2021 to 7.1% in 2025 (GitClear, 2025). Developers are writing more code with AI and cleaning it up less.
The key insight: this is the least amount of slop your codebase will ever have. AI authorship is growing. The only viable strategy is to build systems that make slop safe to ship — not to pretend you can read your way out of it.
Why traditional code review cannot solve the slop problem
Traditional code review assumes a human reads and understands every diff. That assumption is already broken. The Sonar survey found that only 48% of developers always verify AI-assisted code — meaning more than half of AI-generated code reaches production without full human review (Sonar). Senior engineers report spending 20-35% more time reviewing when juniors lean heavily on AI assistants — a hidden tax that makes the bottleneck worse, not better.
Adding more review is a losing arms race because the code generation rate scales faster than any team's review capacity. The old processes — manual review, pre-merge checklists, senior engineer sign-off — were designed for a world where every developer wrote a human-readable number of lines per day. Today, an agent session can produce thousands of lines in minutes.
The answer is not to read more code. It is to build machine-verifiable invariants that catch slop automatically so that the code you don't read is still trustworthy by construction. This echoes a broader principle we explored in When Not to Use AI: The Deterministic-First Rule That Stops AI From Wrecking Your Work — the goal is always to know which parts of your system demand human certainty and which parts can tolerate AI-driven iteration.
The 7-Layer Slop Defense Framework
This framework is a synthesis of engineering practices that work in 2026 — each layer catching a different class of AI-generated defect that traditional review misses. You need all seven; removing any one creates a gap that slop flows through.
| Layer | What it catches | Tools / mechanism | Cost to build |
|---|---|---|---|
| 1. Architecture contract | Structural drift, broken module boundaries | ARCHITECTURE.md + CI invariant checks |
Low (1 file + CI rule) |
| 2. Design-first docs | Bad design decisions shipped as code | Async design doc tool + read receipts | Medium |
| 3. Dependency graph watchdog | Leaky abstractions, circular deps | Dependency visualizer + commit-hook guards | Medium |
| 4. Automated eval pipeline | Hallucinated features, off-by-one logic, inefficiency | Agent generates → agent evaluates → human triages | High but scales |
| 5. Execution-trace navigator | Hidden runtime bugs, performance regressions | Built-in tracing per function; zero-cost in compiled langs | Medium |
| 6. Type system truth layer | Wrong types, unhandled errors, unsafe casts | Strict typing; inferred error types; exhaustive handling | Low (language feature) |
| 7. Agent-first search + run | "What does this actually do?" gaps | Symbolic code search; every function is a runnable CLI command | Medium |
Layer 1: The architecture contract (stop fighting over tools)
The first battle in fighting slop is the battle of standards. Some engineers want to use Claude, some want Codex, some want the latest open-source model they found. Trying to force a single tool creates friction and fails anyway — developers will use whatever helps them move fastest.
The fix: stop standardizing the tool and standardize the invariant. Write a single ARCHITECTURE.md (or CLAUDE.md for Claude Code, or .cursorrules / .cursor/rules/*.mdc for Cursor) that every model can read at session start. According to Claude Code's official documentation, a project CLAUDE.md stored at ./CLAUDE.md or ./.claude/CLAUDE.md is automatically loaded as persistent context at the start of every session, carrying build commands, coding standards, architectural decisions, and naming conventions across all conversations (Claude Code Docs — Memory). Cursor's .cursor/rules/*.mdc files serve the same function with glob-pattern-based activation (Cursor docs).
The rules for a good architecture contract:
- Keep it tiny. If it's longer than a few hundred lines, agents lose track. Only include things that won't change for months or years.
- Describe the layers, not the implementation. For a compiler, list the layers (parser, type-checker, optimizer, codegen). The agent stays inside the layer you asked it to work on.
- Make it model-agnostic. Plain markdown, no tool-specific syntax. Every LLM reads markdown. This matters because which LLM your team uses next quarter is unpredictable — see How to Build a Model-Agnostic AI Agent OS That Survives Model Churn in 2026 for the same lesson applied to agent infrastructure.
- Require the agent to talk to at least one other person before making architectural decisions. This forces a slowdown on design-level changes while allowing speed on implementation.
When this layer is strong, it doesn't matter which AI tool an engineer uses — they all write code within the same architectural constraints because they all read the same contract.
Layer 2: Design docs that people actually read
Everyone knows you should write design docs. Most teams write them in Notion or Google Docs, nobody reads them, and bad design decisions become load-bearing code.
The fix: treat design docs as code, not as Wiki pages. Use a system backed by plain markdown files with a CLI interface (so agents can create and read them). Add versioning and commenting. Most importantly, add a Slack integration that pings a channel every time a design doc is created or updated. This turns design docs from a formality into a real-time feed that engineers actually scan.
The rule that makes this work: code can be slop, writing cannot. If a bad function ships, it can be refactored. If a bad architecture decision ships, it costs weeks. So the design doc — the thinking layer — gets the quality bar that code review used to enforce.
But notifications alone aren't enough. One engineer can ship 10 low-quality design docs a day and drown the channel (this happened). So the final rule: if you ship a design doc, you require people to actually read it. That can mean requiring an acknowledgment, a review window before implementation, or a minimum number of comments before the doc is "approved for implementation." The mechanism matters less than the principle: design docs are the one place where AI-vibe-writing is not acceptable.
Layer 3: The dependency graph watchdog
As a codebase grows, agents add dependencies that break architectural boundaries — a UI module importing a database module, a utility package that depends on a domain type, circular imports that compile but create runtime ordering bugs.
The fix: a dependency visualization tool that renders your internal package graph (with external dependencies shown where relevant). But visualization alone is passive. The real power is automated invariant enforcement: CLI scripts that check the dependency graph on every commit and reject changes that violate the rules.
# Example: CI rule that blocks leaky dependencies
check-dependencies:
- if package "ui" imports from "database", FAIL
- if package "core" imports from "ui", FAIL
- if circular dependency detected, FAIL
- report: which package added the leak, which commit, which agent session
When an agent builds a new package or adds a dependency that's leaky, the CI/CD pipeline (or a pre-commit hook) blocks the merge and tells you exactly where the boundary was broken. This is the difference between detecting architectural drift six months later and preventing it at commit time. Teams using this approach report going months without architectural changes because the guard rails make violations impossible to merge — the architecture stabilizes by construction.
Layer 4: The automated eval pipeline — let agents test agents
This is the highest-leverage layer and the hardest to build. Most AI code quality articles tell you to "review AI code carefully." That doesn't scale. Instead, let agents generate code, and let other agents evaluate, A/B test, and find issues in the generated code.
The pipeline:
- Generate: Agent A writes an implementation from scratch (a function, a module, a feature).
- Record: Capture the full agent transcript — every tool call, every file read, every edit, every error and retry.
- Evaluate: Agent B inspects the transcript and the output. It finds what was good, what was bad, what took three tool calls when it should have taken one, what was incorrect, what was inefficient.
- Triage: Humans review the agent's findings to separate real issues from hallucinations and matters of taste. This is dramatically faster than reading the code itself — you're reviewing a diagnostic report, not a diff.
- Fix: Agent C (or A again) creates fixes for the real issues. Optionally, A/B test language features or patterns by generating multiple versions and deterministically measuring which one took fewer tool calls, produced fewer errors, and achieved the correct outcome.
This transforms code quality from a human bottleneck into a data-driven pipeline. You're no longer guessing which patterns are better — you're measuring them. And the human time shifts from reading code to reviewing diagnostic reports, which is 10x faster per issue found.
The key realization: a compiler has hundreds of invariants (generics, closures, memory allocation, FFI boundaries). Python still has bugs 25 years later. No human can verify all the invariants in a complex system. Agents can — if you build the eval pipeline that lets them. This is closely related to the approach in Simulation Testing for AI Agents: How to Ship LLM Agents 20x Faster Without Waiting on Real Users — letting agents test agents is the same principle whether the subject is code quality or agent behavior.
Layer 5: The execution-trace navigator
When you don't read every line of code, the only way to understand what the code actually does is to watch it run. Execution traces — which function called which, how much time was spent in each part, which branch was taken — become the new "reading the code."
The fix: build a tracing system where every function call is recorded with its inputs, outputs, duration, and caller. The question is not "what does this code say?" but "what did this code do when it ran?"
In interpreted languages (Python, JavaScript/TypeScript), adding tracing to every function call is prohibitively slow. But if you start from first principles — designing the language or toolchain with tracing in mind — the overhead can be effectively zero. This means traces are not just for debugging; they become a first-class way to navigate the codebase.
For practical implementation without rewriting your language:
| Approach | How it works | Overhead | Best for |
|---|---|---|---|
| OpenTelemetry instrumentation | Manual or auto-instrument key functions | Low-moderate | Microservices, APIs |
| eBPF-based tracing (e.g., pixie, parca) | Kernel-level tracing, no code changes | Near-zero | System-level debugging |
| Built-in tracing (BAML/Go-style) | Compiler emits trace hooks per function | Zero (compiled in) | New codebases, agent-generated code |
| Agent-driven tracing | Agent adds temporary trace points, runs test, removes | One-time cost | Investigating specific bugs |
The point: in a codebase where most code was written by an agent, the execution trace is more trustworthy than the source code — the trace cannot lie, but the source can hide bugs in edge cases you never reach.
Layer 6: The type system truth layer
This is the foundation. When the type system is strong enough, it becomes the absolute source of truth that prevents invariants from entering your codebase — no matter which tool wrote the code.
The problem with most popular languages is that they were designed for human productivity, not agent productivity. TypeScript's design goal, per its own documentation, is to "strike a balance between correctness and [human] productivity" — but the asterisk is on human. Features that help humans write code fast (implicit string coercion in sorting, any casts, structural typing with escape hatches) become the exact places where agents create slop. (The broader tension here — human-optimized languages vs. agent-optimized toolchains — also drives the move toward faster, stricter build tooling, which we examined in TypeScript 7 in 2026: The Go Compiler Rewrite That Cuts Build Times by 10x.)
Type features that prevent slop:
- Inferred error types. In a slop-proof type system,
divide(a, b)knows it throwsDivisionByZeroError— without you writing a single annotation.calculate(a, b)which callsdividealso knows it throwsDivisionByZeroError. The compiler infers the error types across the call graph, so you can't accidentally ship an API that throws an unhandled error. - Exhaustive error handling. If you catch an error, the compiler proves you've handled every variant. If you ship an API that promises to never throw, the compiler checks that promise against the actual error types in the call graph — and rejects the code if two errors aren't caught.
- No
anytype. No implicit casts. Types persist at runtime, so an agent can't silently coerce a number to a string and pass it across a boundary. - Cross-language type safety. When a typed function is called from Python (via FFI), the type signature carries over. You can pass lambdas, generics, and closures across the boundary safely. The type system never lies — even across languages.
Rust is the closest mainstream language to this ideal — its Result<T, E> type and exhaustive match on errors make it the gold standard for error handling that agents can't accidentally break. BAML ("Basically A Made-up Language"), an open-source programming language for agents built by Boundary (a Y Combinator W23 startup), implements several of these principles: a type system like Rust, compiled error types, no any, and type persistence at runtime (GitHub: BoundaryML/baml).
For existing codebases, you can approximate this layer with strict TypeScript (strict: true, noImplicitAny, strictNullChecks) or mypy strict mode in Python. It won't be as airtight as a language designed for it — but it closes the biggest gaps.
Layer 7: Agent-first search and runnable functions
The last layer transforms how humans and agents navigate the codebase. Traditional grep-based search returns matching lines — useful, but you still have to read the surrounding context to understand what a function does.
The fix:
- Symbolic code search. Instead of
rg "calculate", a search command that returns: the function signature, all docstrings, the full source, and everywhere the function is called — in a single tool call. This collapses multiple agent round-trips into one. - Run any function instantly. Every function in the codebase is available as a CLI command.
add 2 3runs theaddfunction with parametersa=2, b=3and prints the result. No need to write a test harness or set up a REPL — just run the function and see what happens. - Cross-library queries. Ask "what does this external library's
parsefunction do?" and get the actual source code + docstrings, not a web search result. The code is always the source of truth — docs may lie, READMEs definitely lie, but the code cannot.
This layer matters because agents spend a huge fraction of their tool calls on understanding code — searching, reading, and re-reading files to build context. Each of those calls is a token spend and a latency hit. Agent-first search reduces the number of calls needed, which reduces both cost and the surface area for the agent to get confused and produce slop.
The practical version for existing codebases: use ripgrep (a Rust-based search tool that is faster than grep, respects .gitignore by default, and is the de facto standard for code search in 2026) for the search layer, and build small CLI wrappers around key functions so agents can execute and verify behavior in one step.
How to implement the framework on an existing team
You don't build all seven layers at once. Here's a realistic adoption order:
Phase 1 (Week 1): Layers 1 + 6 — the cheap wins.
- Write your
ARCHITECTURE.md/CLAUDE.md/.cursorrules. Keep it under 200 lines. Describe the module layers and what must not change. - Turn on strict mode in your type checker (
strict: trueintsconfig.json, ormypy --strictfor Python). This catches the largest class of agent slop immediately and costs nothing but config.
Phase 2 (Weeks 2-4): Layers 3 + 7 — the tooling layer.
- Add a dependency-check script to CI. It doesn't need to be fancy — a script that checks import rules between your top-level packages and fails the build on violations. Wire it into your pre-commit hook so agents can't merge leaky code.
- Adopt ripgrep if you haven't already. Write small CLI runners for your most-called internal functions so agents (and humans) can execute and verify in one step.
Phase 3 (Month 2): Layers 2 + 5 — the culture and visibility layer.
- Set up your design doc system: markdown files in a
docs/directory, a CLI for create/comment/version, and a Slack integration that pings on every new doc. Write the read-receipt rule into your team agreement. - Add tracing to your critical-path functions. Start with OpenTelemetry instrumentation on API handlers and database calls. You don't need zero-overhead tracing on day one — you need visibility.
Phase 4 (Month 3): Layer 4 — the eval pipeline.
- This is the big one. Start small: pick one module. Write an agent that generates implementations. Write a second agent that reviews the transcript and flags issues. Have a human triage the findings for a week. Then automate the triage for findings the human consistently validates.
- Once the pipeline works for one module, expand it. The goal is not to replace code review everywhere — it's to build a data-driven quality loop for the modules that change most often.
What this means for you
If you're a solo developer or small team: Layers 1 and 6 are 80% of the value. Write a tight architecture file, turn on strict typing, and you've eliminated the most common slop classes in an afternoon. Layer 7 (ripgrep + runnable functions) is a weekend project that makes every future AI session more productive.
If you're a tech lead or engineering manager: The dependency graph watchdog (Layer 3) is your highest-leverage tool. It turns architectural review from a subjective, slow, human process into an automated, fast, deterministic one. Pair it with the design doc channel (Layer 2) and you've replaced most of the value of traditional code review without the bottleneck. The deeper challenge is recognizing your own role — a point we explored in Why You Are the Bottleneck in Your AI Agent Workflow (and the 4-Part Fix) — the guardrails only help if the human-in-the-loop uses their freed-up time on judgment, not on fighting the AI's output line by line.
If you're building from scratch (new language or greenfield project): This is where Layers 4, 5, and 6 compound. Design the type system with inferred errors and exhaustive handling from day one. Build the execution trace into the language so it's zero-cost. Let agents generate and agents evaluate. You can build complex, correct systems with a tiny team at agent speed — but only if the trust infrastructure is there.
For everyone: Stop optimizing for "I read every line." Start optimizing for "the system catches what I didn't read." The former is a recipe for burnout and a false sense of security. The latter is what actually scales in 2026.
FAQ
Q: Does this framework mean I should stop code review entirely? A: No. It means code review shifts from "read every line for correctness" to "review design decisions, agent evaluation reports, and architectural changes." You're still accountable — but your time goes to the things humans are best at (judgment, taste, domain context) while the type system, dependency guards, and eval pipelines handle the mechanical checks. Some teams will find they no longer need mandatory pre-merge review on every PR; others will keep it for critical paths. The framework gives you the choice.
Q: How much does implementing all 7 layers cost? A: Layers 1 and 6 (architecture file + strict typing) are essentially free — a few hours of work. Layer 3 (dependency checks) is a day of scripting. Layer 7 (ripgrep + CLI runners) is a weekend. Layer 2 (design doc system) is a few days of tooling + a team agreement. Layers 4 and 5 (eval pipeline + tracing) are the real investment — a few weeks of engineering time, but they compound forever. Most teams get 80% of the value from the first four layers under a month.
Q: What's wrong with just adding more code review for AI-generated code? A: It doesn't scale. The Sonar 2026 survey found that 42% of committed code is already AI-generated and developers predict it will reach 65% by 2027. Review capacity is fixed; generation capacity is exponential. You can't hire reviewers fast enough. You need automated guardrails — the type system, the dependency checks, the eval pipeline — that catch slop without a human in the loop for every change.
Q: Can I apply this framework to an existing TypeScript or Python codebase?
A: Yes. Layers 1, 2, 3, 6 (partial), and 7 apply directly and require no language change. Layer 6 in TypeScript means strict: true + noImplicitAny + strictNullChecks — you lose some of the inferred-error-type guarantees but close the biggest gaps. In Python, mypy --strict with --disallow-untyped-defs achieves a similar effect. Layers 4 and 5 (eval pipeline + tracing) work in any language. The full power of Layer 6 (inferred error types across the call graph) requires a language designed for it, but strict mode in your existing language catches the most common classes of slop immediately.
Q: What is BAML and should I switch to it?
A: BAML ("Basically A Made-up Language") is an open-source programming language designed specifically for agent-generated code, built by Boundary (YC W23). It has a type system inspired by Rust, compiles faster than Go, has no any type, infers error types, and is designed to run alongside any host language (Python, TypeScript, Go) (GitHub: BoundaryML/baml). You don't need to switch to it to apply this framework — strict typing in your existing language covers most of the same ground. But if you're building a new agent-heavy project from scratch, it's worth evaluating as your "AI module" language while keeping the rest of your stack in the language your team already knows.
Q: How do I stop individual engineers from using incompatible AI tools?
A: Don't try. Standardize the invariant (the architecture contract), not the tool. If every AI assistant reads the same ARCHITECTURE.md / CLAUDE.md / .cursorrules at session start and the CI pipeline enforces the same dependency and type rules on every commit, it doesn't matter whether engineer A uses Claude and engineer B uses Codex or Cursor. The code converges to the same constraints. Fighting over which tool to mandate is wasted energy in 2026.
Discussion
0 comments