Verdict: The AI products that win funding and survive production are not the ones that perform best in a demo — they are the ones that pass the "memo test." A demo's job is to make a room say "oh" for five minutes. A memo's job is to survive an argument from people whose entire job is to not be impressed. If you build for the second scenario from day one, you fix the trust problems that kill products in diligence rooms, procurement reviews, and customer tribunals and you build the same muscles investors look for before writing a check. This is the framework for doing both, drawn from how 200+ investment committee meetings actually work and verified against the real-world incidents that shaped the 2023–2026 AI liability landscape.
Last verified: 2026-07-31 · Most AI products are built for the demo; almost none survive the memo · The five fixes are plumbing, not brainpower · Volatile facts (regulations, rulings) re-checked monthly.
The two machines everyone confuses
What is the difference between a demo and a memo? A demo produces one clean document in and one fluent answer out; its whole job is to make a room go "oh" for five minutes. A memo is the real document a committee races before real money moves — hundreds of pages, filings, transcripts, half the sources disagreeing with each other, someone's rushed notes from a call last Tuesday. Its job is not to make you go "oh." Its job is to survive an argument.
The gap between those two machines is where nearly every AI product fails. Ask your phone to summarize a long email: that is a demo. It only has to sound plausible for ten seconds. Now imagine that same phone has to stand in front of a bank and defend out loud why you deserve a mortgage. Suddenly plausible is not enough. Now it has to be right and it has to prove it. That second situation is what a memo actually is — and it is what every customer, procurement team, and investor eventually turns your product into.
The most expensive typo in AI history
How much can one unchecked AI sentence cost? On February 6, 2023, Google posted a promotional video for its new AI assistant, Bard. Asked what a child should know about the James Webb Space Telescope, Bard answered that the telescope "took the very first pictures of a planet outside of our own solar system." That was wrong. The first direct image of an exoplanet was captured by the European Southern Observatory's Very Large Telescope in 2004 — two decades before JWST launched (NASA Exoplanets; New Scientist, 8 Feb 2023).
When Reuters flagged the error the next morning, Alphabet's stock fell 7.7% on the day, wiping roughly $100 billion in market value in hours (CNN, 8 Feb 2023; NPR, 9 Feb 2023). That is $100 billion gone because of one unchecked sentence. The demo failed the memo test the moment real money was watching — and real money is always watching.
The lesson is not "Google messed up." The lesson is that in an AI world, every sentence is a memo sentence. There is no safe demo anymore. The same quiet, predictable, trust-breaking moments happen six smaller ways inside your product every day. Let's go through them.
The six ways trust quietly breaks
1. Which source do you believe?
Why does treating all sources as equal break AI trust? Because in real documents, a number from an audited filing is your accountant speaking under oath; a number from an analyst's note is a friend at a party; and a number from someone's internal group chat is the thing you overheard in an elevator. Most retrieval-augmented generation (RAG) systems cannot tell them apart. They grab whichever text chunk sits closest to the query and hand it over like gospel.
The failure mode is not theoretical. In one real diligence room, an expensive AI tool confidently pulled a revenue figure from a group chat — a rough guess someone had texted six months earlier — while the real, audited number sat three rows away in the actual filings. The AI reached for the chat version because the phrasing sounded more confident and enthusiastic. If your retrieval pipeline cannot tell an accountant under oath from a rumor in a group chat, it is not ready for real money.
The fix is source-level trust scoring. Every chunk that enters the context window should carry a provenance tag — audited, regulatory filing, analyst estimate, internal communication, web scrape — and the model's output should surface that tier alongside the number. A viewer should see "Revenue: $4.2B (source: audited 10-K, FY2025)" not "Revenue: $4.2B." The provenance is the product.
2. Do the numbers agree with each other?
Why do internal number inconsistencies destroy credibility? Because the committee does not care about the missing 0.6%. They care about what it implies: if this person did not check the easy math, what did they not check on the hard stuff?
A memo that says "revenue growth 18%" on page one and "17.4%" in a tucked-away table on page 11 does not fail because of the 0.6% gap. It fails because the gap tells the room the document was not checked end-to-end. The industrial-strength version of this failure is Zillow. In November 2021, the real-estate giant shut down its algorithmic home-buying unit, Zillow Offers, after its pricing model confidently overpaid for houses the market disagreed with. The company took a write-down of more than $500 million (Stanford GSB cites $569 million, approximately $30,000 per home in inventory), laid off roughly 2,000 people — about 25% of staff — and wound the entire unit down (GeekWire, 2 Nov 2021; Stanford GSB, 9 Dec 2021).
The model was not stupid. The model was unsupervised. Nobody built the boring machinery that forces the numbers to keep agreeing with reality after lunch on day three. Fluent and confident is not the same as right.
3. Do you hide contradictions or show them?
Why is a contradiction a gift, not a bug? Because when a CEO says one revenue figure on an earnings call and the official filing shows a different one, the gap is the single most interesting thing in the real story. Real diligence looks for that gap. AI does the opposite — it is trained to sound smooth and helpful, so when it hears a conflict it quietly picks whichever version reads nicer and moves on. You never learn there was a disagreement.
In one actual committee meeting, a CEO's stated number and the filing number were meaningfully different. Nobody flagged it. Everyone used the nicer one. The discrepancy was caught only because one person happened to have both documents open at the same time. Pure luck. Luck is not a control. Your job as a builder is not to resolve the argument; it is to make sure the argument happens in front of a human instead of quietly vanishing inside the box. For a deeper treatment of how to architect that surfacing across multi-agent systems, see our guide to verification skills for multi-agent graph workflows.
4. Is that a fact or a guess?
Why does blending facts and estimates destroy a decision? Because a committee's entire job is to agree with the guesses while trusting the facts. If your system melts them into one smooth sentence — "the company will likely receive approval next quarter" — the committee cannot find the seams, and all they can do is approve or reject the whole document as if every part is equally solid.
In one real case, a draft that started with "approval expected soon" drifted across three rewrites into "approval received" — nobody lied, the guess just wore its fact costume a little longer each time until nobody remembered it had started as a guess. The approval did not arrive on schedule. That was an uncomfortable phone call.
The fix is embarrassingly cheap. Label your guesses. A tag, a color, a confidence score, anything that survives being copy-pasted into someone else's slides three weeks later. Facts and estimates must live in visibly separate boxes on the page.
5. Can you prove it in 30 seconds?
What is the 30-second click-through test? When someone points at a sentence and says "show me where this comes from," you either click once and land on the exact source paragraph, or you open seven browser tabs and start swiping. There is no middle ground. This test earned its name because a lawyer named Steven Schwartz failed it publicly in May 2023.
Schwartz, a 30-year veteran attorney at Levidow, Levidow & Oberman in New York, used ChatGPT to research a legal brief for Mata v. Avianca. ChatGPT returned six case citations — proper formatting, realistic court names, plausible docket numbers, complete with judicial reasoning. Every single case was fabricated. Before filing, Schwartz got suspicious and asked the chatbot if the cases were real. The chatbot said yes. (That is like asking the person who sold you a watch whether the watch is real.) Judge Kevin Castel found all six citations were "bogus judicial decisions with bogus quotes and bogus internal citations," fined Schwartz $5,000 in June 2023, and required him to notify every falsely named judge (CNN, 27 May 2023; CNBC, 22 June 2023).
The fake cases even had real-sounding names and page numbers. The AI did not just lie — it cited itself beautifully. Wrong, but beautifully. If you remember one rule from this whole framework: the click-through is the product. Everything else is well-written packaging.
6. Whose name is actually on the decision?
Can you outsource accountability to your own software? No. On February 14, 2024, the British Columbia Civil Resolution Tribunal answered that question in Moffatt v. Air Canada (2024 BCCRT 149). Jake Moffatt's grandmother had died. He asked Air Canada's website chatbot about bereavement fares, and the chatbot told him he could book a full-price ticket and claim abereavement discount within 90 days after travel. That policy did not exist. Air Canada's actual rule required the bereavement fare to be requested before booking.
When Moffatt claimed the refund and was denied, he took the airline to the tribunal. Air Canada's defense was, in effect, that the chatbot was "a separate legal entity responsible for its own actions." The tribunal rejected that argument entirely. Tribunal member Christopher Rivers found Air Canada liable for negligent misrepresentation and awarded Moffatt CAN$812.02 in damages and fees. The ruling established that companies are responsible for what their AI says, even when the AI invents things from whole cloth (CanLII commentary, 2025; McCarthy Tétrault analysis). The airline quietly retired the chatbot afterward.
At the bottom of every real decision, a human signs. If your architecture does not have a fundable human at the end of it, you have not built a product. You have built an excuse generator. Build the AI around that accountable person, not instead of them. For the governance patterns that enforce this across an organization's growing agent fleet, see our AI agent sprawl governance guide.
The five-part framework that fixes it
These are the five requirements serious diligence now demands of any AI product before it touches a live deal. Notice what is not on the list: a smarter model, a bigger context window, a higher benchmark score. None of these is a brainpower problem. All five are plumbing and honesty problems.
| # | Requirement | What it cures | Implementation signal |
|---|---|---|---|
| 1 | Every claim comes with a receipt | The 30-second test (failure #5) | Each sentence links straight to its source paragraph with source trust tier attached — not a citation tab at the end |
| 2 | Facts and guesses stay visibly separate | The fact-estimate melt (failure #4) | At a glance, the viewer sees what is proven and what is someone's best estimate — a tag, color, or confidence score that survives copy-paste |
| 3 | Numbers agree with each other automatically | The inconsistency decay (failure #2) | The system refuses to ship a document where the figures do not match — no human checking at 2 a.m. |
| 4 | Contradictions get surfaced, never smoothed | The hidden conflict (failure #3) | When sources disagree, the system raises its hand instead of picking the prettier answer |
| 5 | A real, logged human approval gate | The accountability vacuum (failure #6) | Who reviewed what, when they signed, and what changed since — that log is the audit trail |
The pattern across all five: each is a direct cure for one of the six trust failures. The winners in this category — and the ones that close enterprise deals — will not win on benchmark points. They win because a tired, skeptical finance person can trust their output at 11 p.m. without opening seven tabs.
How to implement the framework in your AI product
How do you actually implement source-tier provenance? Start by labeling every chunk in your retrieval pipeline with a source_kind field. A regulatory filing gets source_kind: regulatory. An analyst note gets source_kind: analyst. A Slack message gets source_kind: informal. When the model generates an answer, it must include the source_kind alongside the citation. Render this as a visible badge on the page — not buried in a citation list. This is cheap to build and it is the single highest-leverage trust signal in the whole stack; for the architectural patterns that make this automatic across many agents, see our guide to building a model-agnostic AI agent OS.
How do you enforce number agreement automatically? Pair the generation step with a validation pass. After the model assembles a memo, extract every numeric figure with a regex or structured-output constraint, then cross-check that the same metric is reported the same way everywhere it appears in the document. If the revenue growth is "18%" on page one and "17.4%" on page eleven, the system blocks the draft and returns it for reconciliation. This is a 50-line script. Most teams skip it because it is boring. That is why most teams fail the memo test.
How do you surface contradictions instead of smoothing them? When two retrieved sources conflict, do not instruct the model to "reconcile" them. Instead: generate both versions side by side, label them as conflicting, and flag the conflict for human review. The model's job is to present the disagreement; the human's job is to resolve it. This is the single hardest change for teams trained to optimize for "helpfulness," because it deliberately produces an answer that looks less smooth — and that is the point. For why "helpfulness" as a design target is itself the failure mode, see our piece on why AI fails in the enterprise.
How do you separate facts from estimates legibly? Use a two-tier output schema. Every sentence the model generates carries a confidence property: proven (backed by a primary source), estimated (model-derived or analyst-projected), or unverified (no source found). Render this with a visual cue — color, weight, or a small icon — so the reader can scan the page and instantly distinguish what is load-bearing from what is a working assumption. The key design constraint: the label must survive being copied into someone else's deck three weeks later. Plain-text tags survive; color-only labels do not.
How do you build a real human approval gate? The gate is not a button labeled "Approve." It is a logged event with three required fields: who approved, what version they approved, and what changed since the previous version. This means your product must keep an immutable revision history — not just "current state" but a diff-able trail of every edit. That trail is what a diligence team, an auditor, or a tribunal will ask for. If the trail does not exist, you cannot prove accountability after the fact — and Moffatt v. Air Canada says you are liable anyway.
What this means for you
For builders (AI product teams): Stop optimizing for the demo and start optimizing for the memo. The five framework requirements are not features you bolt on at the end — they are architectural decisions that have to be in the retrieval schema, the generation constraints, and the output rendering from day one. A product that adds provenance tags in v2 will always leak un-tagged claims from v1. Build them in at the data layer.
For founders raising money: Your pitch deck becomes a memo the moment you leave the room. Every number you say out loud gets checked against your data room. The same five requirements that make your product trustworthy to a procurement team are the ones that make your diligence clean to an investor. Build the trust layer before the raise, not during it — the commercial infrastructure gaps a venture diligence team finds are exactly the same gaps a procurement team finds, and fixing them early costs less than fixing them under deadline pressure.
For small businesses buying AI tools: When you evaluate an AI product, apply the memo test before you buy. Ask the vendor: "Show me where this claim comes from" — and time the click. Ask: "How do you separate facts from estimates?" Ask: "Who signs off on an output, and where is that logged?" If they cannot answer in 30 seconds, you are looking at a demo, not a product. The tools that pass this test are the ones that will not put you on the wrong end of a tribunal in 2027.
FAQ
Q: What is the memo-first framework for AI products? A: It is a six-failure, five-fix model for building AI products that survive real scrutiny — the kind investors, procurement teams, and tribunals apply. The five fixes (receipts on every claim, facts vs. guesses separated, automatic number agreement, surfaced contradictions, logged human approval) are plumbing and honesty requirements, not model capability upgrades.
Q: Can AI ever fully eliminate hallucinations? A: No. Current LLM-based systems cannot fully eliminate hallucinations. The honest approach is to reduce risk through retrieval grounding, provenance tagging, confidence scoring, and human oversight — and to label anything you cannot verify rather than present it as fact. The goal is a product a tired, skeptical professional can trust at 11 p.m. without opening seven tabs.
Q: What is the 30-second click-through test? A: When someone points at a sentence in your AI output and says "show me where this comes from," you either click once and land on the exact source paragraph, or you fail. The test is named for the Mata v. Avianca case where a lawyer's AI-generated citations could not be traced to any real case — every citation was fabricated, and the lawyer was sanctioned $5,000 in June 2023.
Q: Are companies liable for what their AI chatbot says? A: Yes. In Moffatt v. Air Canada (February 2024, 2024 BCCRT 149), the British Columbia Civil Resolution Tribunal ruled that a company is responsible for the representations its chatbot makes, even when the chatbot invents policies that do not exist. Air Canada's argument that the chatbot was "a separate legal entity" was rejected, and the airline was found liable for negligent misrepresentation.
Q: How much did Google's Bard demo error cost in 2023? A: Google's promotional video for Bard incorrectly stated that the James Webb Space Telescope took the first images of an exoplanet; the first such image was actually captured by the Very Large Telescope in 2004. When Reuters flagged the error on February 8, 2023, Alphabet (Google's parent) fell 7.7% on the day, wiping approximately $100 billion in market value.
Q: What is the difference between building for the demo and building for the memo? A: Building for the demo means optimizing the product to perform well in a five-minute controlled presentation where one clean input produces one fluent output. Building for the memo means optimizing for the real-world scenario where the output has to survive an argument from people whose job is to not be impressed — auditors, investment committees, procurement reviewers, and tribunals. The first sells the pilot; the second closes the deal.
Discussion
0 comments