Twenty-six percent.
That was the true positive rate of OpenAI's own AI-text classifier at a nine percent false positive rate โ two numbers published in the same July 2023 note that announced the tool's retirement. Read them together, because together they are the whole argument: at the operating point its own maker chose, the machine was wrong roughly three times more often than it was right, and every one of those wrong answers landed on a person with a name, a deadline, and no way to prove a negative.
I keep that pair on a sticky note above my monitor the way other analysts keep a volatility index. It is my harm quota. It reminds me that in the business of adjudicating whether a paragraph came from a human or a model, the accuracy curve is not a technical curiosity โ it is a ledger of accusations, and in a bull market nobody reads the ledger.
So when a research collaboration between Google DeepMind and the University of Pennsylvania surfaces with the framing that its detector will "show its work" โ that it will not merely flag a document as machine-written but explain why โ my first instinct is not excitement about interpretability. It is to ask who is about to be handed a receipt they cannot contest. I have watched this maneuver before, in this industry and outside it. It never arrives shaped like a weapon. It arrives shaped like transparency.
There is a ticker for this now. Several, actually. This cycle has produced a cluster of tokens trading against the thesis that AI-generated content will be verified, attested, scored, certified โ pick your verb โ and the price action implies the verification problem is being solved in real time. It is not. What is being solved is a distribution problem: the same statistically fragile instrument is being repackaged with a friendlier interface and pushed into institutions that have already decided they need an answer, and that will pay for the appearance of one.
The detection problem has not improved. The sales pitch has. And the pitch is now written in the vocabulary my own industry invented โ auditable, verifiable, transparent, trustless โ borrowed without the obligations that make those words mean anything at all.
The modern detection industry has a short and unusually well-documented failure record, and any honest reading of it starts in the same place: a detector is a classifier trained on the artifacts of one generation process, deployed against an adversary who controls the input and who is rewarded for defeating it.
In January 2020, OpenAI shipped a GPT-2 output detector that reported roughly ninety-five percent accuracy on its own model's samples. It degraded immediately under paraphrase, under truncation, under a second model's output, under translation. Three years later the company shipped a general classifier, and by July 2023 it had pulled it, publishing that 26/9 pair on the way out. The obituary was unusually candid and almost nobody read it.
In parallel, the commercial layer grew. Turnitin's AI writing indicator, launched in April 2023 and deployed across tens of thousands of institutions, claimed sub-one-percent false positives โ a claim that is definitionally true only if you define the population in which you measured it, and which collapsed under scrutiny the moment linguists pointed out that the population in question excluded exactly the writers most likely to be misread. The Patterns paper by Liang and coauthors found that across seven popular detectors, an average of 61 percent of TOEFL essays written by non-native English speakers were labeled machine-generated. One commercial tool flagged nearly all of them. That is not an edge case. That is a structural bias produced by the fact that non-native academic English and machine English share a distribution: low perplexity, low burstiness, predictable transitions, standardized connective tissue.
The theoretical case got settled the same year. Sadasivan and coauthors argued in "Can AI-Generated Text be Reliably Detected?" that recursive paraphrasing โ not adversarial research, just a student with a second model and a free afternoon โ drives detectors toward random performance, and tied that empirical result to the cryptographic literature on watermarking: without a secret shared between generator and detector, no reliable distinction is possible against an adversary who can query. That is a strong claim and it has held up.
Set that against the three technical paths, which are almost never separated in press coverage and must be separated now, because everything downstream depends on it.
Path one: watermarks. A statistical signal embedded at generation time by biasing token sampling against a keyed pseudo-random partition of the vocabulary, verified later by a hypothesis test that yields a p-value. This is genuinely the strongest path, because it is not inference about a text โ it is a test of a null hypothesis about a sampling procedure. It produces real evidence with a real error model.
Path two: classifiers and perplexity heuristics. Statistical inference about the distributional fingerprints of a text. This is the path with the 26/9 pair, the 61 percent TOEFL number, and no principled way to generalize across models it was not trained on.
Path three: provenance metadata. C2PA-style content credentials, cryptographic signatures from capture device or generation model, hash-chained and optionally anchored to a public ledger. This is not detection at all. It is attestation of origin, and it makes a completely different claim.
Three paths, three epistemics, three failure modes, three different products. The source coverage โ a two-to-four-paragraph item on a crypto outlet's AI page, rewritten from a paper abstract โ collapses them into one word: detection. It gives no false positive rate, no true positive rate, no test set, no cross-model generalization result, no robustness number under paraphrase. It gives a framing. For a technology whose only meaningful evaluation dimensions are precisely the ones omitted, that silence is not a gap in the reporting. That silence is the reporting.
And it was published to a crypto audience for a reason. The word doing the work here is not "detection." It is "auditable." That word is the native currency of this industry โ it is the reason we care about consensus, about proofs, about attestations, about anything that can be checked by a stranger without trusting an intermediary. Which is exactly why the crypto audience is the most vulnerable audience on earth to this particular reframing. We have a trained reflex: auditable means trustworthy. It does not. And the gap between those two words is where the next twelve months of damage will happen.
Here is the first thing I want to put in bold, because it is the thing the framing is engineered to hide: explainability and accuracy are orthogonal axes. Attaching a rationale, a highlighted evidence span, or a chain of reasoning to a verdict does not change the verdict's error rate by a single basis point. It changes the verdict's persuasiveness. And in the specific context of adjudication โ a professor deciding on a disciplinary referral, a hiring manager binning a rรฉsumรฉ, an editor killing a freelance pitch โ persuasiveness is the variable that determines whether the error reaches a human being.
Worse, the rationale may be fiction. The interpretability literature has a name for this: unfaithful rationales. A language model asked to justify a classification will generate a plausible narrative for the classification, not a causal account of the signal that produced it. The explanation is downstream of the label, not upstream. Anyone who has audited a smart contract after reading its documentation knows this shape intimately โ the comment block says the function is reentrancy-guarded, the bytecode says otherwise, and the comment block was written by the same person who wrote the bug, in good faith, believing it.
So when a detector "shows its work," the honest question is not whether the work is legible. It is whether the work is load-bearing. And there is no public evidence that this work is load-bearing, because there is no public evidence of anything yet โ no faithfulness eval, no ablation showing that removing the cited evidence flips the verdict.
Now the arithmetic that nobody in the announcement cycle runs. It is the piece of analysis I would want a decision-maker to read before signing a procurement order, and it takes four lines.
Take a detector with 90 percent sensitivity and 95 percent specificity โ numbers that would be considered respectable, and that are considerably better than what has been publicly demonstrated. Now deploy it in two rooms.
Room one: a large professional context where one percent of the documents are fully machine-generated. Per ten thousand documents: one hundred true positives exist, of which the tool catches ninety. Of the 9,900 human documents, five percent โ 495 โ are flagged. Total flags: 585. Of those, 90 are correct. Positive predictive value: 15.4 percent. Roughly six out of every seven people accused are innocent.
Room two: a first-year seminar in 2026 where ten percent of submissions are substantially machine-generated. Per ten thousand: 1,000 positives, 900 caught. False positives: 450. Total flags: 1,350, of which 900 are correct. Positive predictive value: 66.7 percent. One in three accused is innocent.
Same detector. Same weights. Same vendor. Same marketing. The accuracy of the tool is not a property of the tool. The buyer sets the accuracy by choosing where to point it, and the buyer is never told this. Vendors report sensitivity and specificity because those are properties of the model. They do not report positive predictive value because PPV requires the base rate, and the base rate is a fact about the customer's population โ which means the customer can be sold a 15 percent instrument believing they bought a 90 percent one, and there is no lie anywhere in the transaction.
Scale that. A platform reviewing ten million documents a day at a one percent false positive rate generates one hundred thousand wrongful accusations per day. Not per year. Per day. At that volume, the false positive rate stops being a technical parameter and becomes a budget line: the harm quota, the number of people your institution has pre-authorized itself to damage.
I have spent enough time inside slashing mechanics to find this shape familiar, and the comparison is not decorative. Ethereum's slashing design is the most carefully engineered wrongful-punishment system in production, and it works because of four properties: slashing conditions are objectively checkable by any node; evidence is public and reproducible; penalties are correlated so that a mass false-slashing event is self-defeating for the attacker; and the whole apparatus sits under a social consensus layer that can, in extremis, override the code. AI text detection is a slashing mechanism with no fraud proof. The evidence is a probability emitted by a model whose weights the accused cannot inspect, the threshold is proprietary, the verdict is not reproducible across versions, and there is no fork, no governance vote, no escalation path that restores a dismissed student's semester.
And notice the direction of the inference. In crypto we solve the sybil problem by making identity costly โ stake, biometrics, social graph, proof-of-personhood. Detection tries to solve the same problem by inference from a single artifact. Inference is the cheapest of the three and the only one with an error rate that scales with the population's writing style. We knew this. We built entire protocols around the premise that cheap inference about identity is a losing game.
The second structural point is political economy, and it is the one I most want the open-weights crowd to internalize, because I do not think they have.
Watermarking requires control of the sampling process at generation time. That control exists only where the generator is operated by someone who holds the key. Fine-tuning on watermarked outputs degrades the signal; open weights can have the watermark stripped, replaced, or never applied in the first place; a local model on a laptop is structurally invisible to any keyed detection scheme. Therefore: a regulatory regime that mandates "AI-generated content must be machine-detectable" is, functionally, a mandate that compliant generation be closed. Open weights become non-compliant by construction โ not through malice, but through geometry.
We have run this experiment. It is called the Travel Rule. A compliance regime designed around identifiable intermediaries does not merely fail to accommodate self-custody; it renders self-custody presumptively illegitimate, and the burden of proof migrates to the user. The AI version writes itself: if detectability is the compliance standard, the labs that operate their own inference get a certificate, and the labs that ship weights get a liability.
The commercial logic underneath is the oldest playbook in platform strategy. Detection is a complement to generation. Whoever gives detection away for free destroys the margin of everyone selling it, and captures the standard. DeepMind's output here is not a product with a price; it is a standard-setting move dressed as research, and its return is realized as enterprise compliance premium on the cloud that hosts the generator, not as revenue from a detector SKU. The third-party detection vendors โ the Turnitin tier, the GPTZero tier โ have their pricing power built on institutional fear rather than technical precision, and free supply from the entity that also owns the generator compresses that power to nothing. Their valuation logic is about to be forced from "accuracy" to "channel," and channel is a worse business.
Third structural point, and this is where my own industry is about to embarrass itself: detection, provenance, and audit are three different accountability objects and the market is collapsing them into one word.
Provenance โ content credentials, signed capture, hash-chained manifests, optionally anchored on-chain โ answers exactly one question: did this file come from a device or model holding this key. It does not answer whether the content is true. It does not answer whether the human directed it. A signed photograph of a staged scene is perfectly attested and completely misleading. Presence of credentials is not evidence of authenticity; absence of credentials is not evidence of fabrication, because the overwhelming majority of the world's content is produced by tools that will never sign anything.
Audit โ model cards, evaluation suites, training-data documentation, third-party red-teaming โ answers a different question: is this system behaving within its declared bounds. That is the layer where actual AI accountability lives, and a text detector contributes nothing to it.
Detection answers a third question: what is the probability this artifact was produced by a class of models, given the artifacts of that class. It is the weakest of the three, and it is the one being sold.
The on-chain version of this deserves specific deflation because I have watched three pitches this quarter that blur it. Anchoring a C2PA hash to a public ledger gives you timestamped existence and non-repudiation of a signing key. That is real and it is narrow. It does not survive re-encoding โ screenshot a credentialed image, and the credential is gone; transcode a signed video for a social platform's compression pipeline, and the manifest detaches. It does not tell you the signer was honest. It cannot cover unsigned generation. It is, in the precise sense, an oracle problem: the chain can only attest what was fed to it, and authorship is an off-chain fact about a human's behavior, not an on-chain fact about a file's bytes. You can put a signature on a document. You cannot put a signature on the intention behind it.
Which brings me to the part of this that I think is genuinely counter-intuitive, and where I expect most of my readers to disagree with me.
The consensus position in the provenance camp โ my camp, structurally โ is that detection is a bad substitute for attestation and that the correct answer to synthetic media is cryptographic origin. I hold that view. But it contains a mirror-image error that we are about to make in public, and I would rather flag it now than litigate it later.
If we build decentralized detection โ staked oracles scoring content authenticity, DAOs adjudicating disputes, immutable registries of authorship โ we do not escape the false positive externality. We amplify it. A probabilistic accusation that has been notarized, made censorship-resistant, and given an appeal process consisting of a token-weighted governance vote is strictly worse for the accused than a probabilistic accusation made by a vendor who can be sued. Immutability is a virtue when the underlying claim is a fact and a bug when the underlying claim is a probability. We learned this with oracles. We are going to relearn it with authorship.
And there is a second-order effect that almost nobody in this space has priced: a perfect provenance registry is an authorship panopticon. Every draft signed, every edit logged, every sentence attributable to a key. We have built this before, in other contexts, and called it transparency, and then discovered that permanent attribution is a censorship primitive and a recruiting tool and, eventually, a pricing mechanism. The person who cannot write an unsigned sentence cannot think an unrecorded thought. Constructing new myths from the ashes of Luna taught me that the stories we build to restore trust are the same stories that get used to allocate blame โ and the more technically elegant the story, the harder it is to argue with when it is pointed at you.
Here is the contrarian read on the technology itself, steelmanned. Suppose the DeepMindโUPenn work is watermark-lineage rather than classifier-lineage โ that "showing its work" means emitting a calibrated statistical test rather than a legible rationale. In that case the technology is sound within its domain, and the domain is exactly one thing: self-verification by the generator. A lab attesting that its own outputs are detectable, for platform integrity and enterprise compliance. That is a real product with a real buyer and a real public benefit. It is also the precise opposite of forensics against arbitrary third-party text, which is what the coverage implies and what every institution will assume it purchased. The gap between those two use cases is the gap between a smoke alarm and a DNA test, and both are being sold under the word "detection."
Which leads to the sharpest contrarian point I have, and the one that should reframe how anyone allocates capital against this narrative: if detection ever actually worked, it would destroy its own market.
The value of a detection signal is proportional to the ambiguity it resolves. Universal, reliable, cheap detection of machine authorship would render the question boring โ the way "was this typed on a Mac?" is boring โ and the moment authorship stops being ambiguous, it stops being a signal of anything. Institutions would not stop evaluating people. They would simply stop evaluating artifacts and start evaluating processes: live defense, version history, keystroke provenance, supervised drafting, oral examination, the slow return of the viva. The detector is a transitional product whose success is its own obsolescence, and the rational institutional buyer in 2026 is not buying detection. They are buying process infrastructure โ the audit trail of how a thing was made โ because that is the asset that survives the detector's failure and the registry's overreach alike.
I have watched this pattern of self-cannibalizing narrative before, in a market where the product's success would have invalidated the thesis that sold it. It ended with a three-month post-mortem and a lot of people insisting they had seen it coming. The tell was the same then as it is now: the pitch described a world in which the problem was solved, and nobody asked what the pitch's own product would be worth in that world.
So what do I actually watch, and what would change my mind?
Four signals, in order of how much they would move me. Whether this ships as weights or as an endpoint โ an open release with an interoperable evidence format would be a genuine public good and would falsify my read that this is standard-setting; a Vertex-only endpoint confirms it. Whether the tool's output is a calibrated p-value with a published threshold or a confidence score with a proprietary cutoff โ the first is checkable by a stranger, the second is not, and only the first deserves the word auditable. Whether it interoperates with C2PA or invents a private schema, because a private schema in a compliance market is a toll booth. And whether the first wrongful-termination or academic-dismissal case that cites a detector output arrives before or after the first regulator writes "detectability" into statute instead of "labeling" โ because those two words are the difference between a disclosure obligation and a licensing regime for closed models.
I am not against detection. I am against the specific move being made here, which is to take a tool with a documented failure mode, attach an explanation to it, and route it into high-stakes decisions on the strength of a word โ auditable โ that we in this industry spent a decade teaching the world to trust. The word was earned by systems where any stranger can re-run the check and get the same answer. This is not that. A detector that shows its work is still a detector. The work is the pitch, not the proof.
So the next time a machine tells you it can show its work, ask the question I ask every time an audit report crosses my desk with a beautiful narrative and no reproducible test vector: whose work is it showing โ the model's, the vendor's, or the institution's alibi? Because the market has already decided the answer it wants, the tokens are already pricing it, and the ledger of accusations is already open. The only thing still unwritten is the column where the false positives get counted, and the only question left is who is going to volunteer to be in it.