How AI Systems Strip Provenance and Why It Matters
Speaker John Ash
Every large language model is built on the same basic transaction: take in human work with identifiable authors and sources, transform it through a latent space, and output it with no attribution trail. The inputs have provenance. The outputs do not. The transformation in between is not incidental. It is the mechanism by which authorship is dissolved.
The Washing Machine
When a model like Claude or GPT produces a response, that response is drawn from a probability distribution shaped by its training data. The training data consists of text written by specific people: researchers, journalists, novelists, engineers, philosophers. Each of those people had names, had contexts, had intellectual histories that led them to write what they wrote. The embedding process strips all of that. Ideas, frameworks, phrasings, and structures are compressed into vector representations where individual authorship becomes statistically irrecoverable. What comes out the other end is presented as the model’s output, or more precisely, as the output of the corporation that owns the model.
Here is what makes this a choice rather than a constraint. The entire architecture of a language model is built on embeddings. Word embeddings encode semantic meaning. Positional embeddings encode where a token sits in a sequence. These are well-understood, standard components. There is nothing, technically, that prevents the same dimensional framework from being applied to source. A source embedding would encode who wrote a piece of training data. A timestamp embedding would encode when it was written. These are the same mathematical objects, the same dimensional properties, applied to different aspects of the data. Transformers already proved that position and meaning can be integrated and attended over jointly. Source and time can be integrated the same way.
The research foundations for this are established. The engineering is tractable. The architecture has even been built and tested. Ablation studies have been run. It is not theoretical. It is proven. But no large corporation has built it, because the second a company demonstrates that source attribution is possible, they become liable under existing copyright law. An independent researcher can be sitting on the solution and it does nothing as long as they have no resources, no connections, and no distribution. The legal defense of the entire industry rests on the implicit claim that attribution through latent space is impossible, that the transformation necessarily destroys provenance. This is a lie. It is a lie that is enormously convenient, and every company in the space is incentivized to maintain it. It is entirely possible that engineers inside these companies have raised this possibility and been told not to pursue it. You do not build the tool that makes your own business model illegal.
This is not a side effect. It is structurally identical to plagiarism, except that the transformation through high-dimensional embedding space makes it invisible by any existing legal definition. You cannot point to a specific stolen sentence because the surface form has been destroyed. But the intellectual substance, the actual ideas and their relationships, survives the transformation intact. The latent space is the washing machine. Dirty attribution goes in. Clean, unattributed output comes out.
The irony of all this is that this very concept has provenance. The phrase “laundering plagiarism,” which gives this piece its title, came from music critic Anthony Fantano, who in a 2024 video reviewing an AI music app that was generating near-identical copies of artists’ work put it simply: “this whole artificial intelligence thing and having it be trained on other artist songs… it’s just plagiarism with extra steps.” It would be strange, in an article about the failure to attribute, to not attribute that. You understand the argument in this section better because of Fantano’s framing. But I am also folding into it my own ideas, my own extensions, my own framework. So what percentage of this section is mine and what percentage is his? These are important questions, and as a human writer I try to work to include attribution where I can see it. But the truth is most people do not cite their sources. Most people do not even know when they have been influenced. People do not know where their patterns come from. They cannot trace the lineage of their own thinking with any precision. But a model can, if we change the math. That is the core of this entire argument. Humans fail at attribution because it is cognitively expensive and often invisible to us. Machines have no such excuse. The provenance is in the weights. It is in the embeddings. It is recoverable. The only reason it is not recovered is because no one who profits from the current arrangement wants it to be.
The Double Extraction
The provenance problem runs in two directions. In the first direction, the model absorbs the work of thousands of authors during training and re-presents their ideas without credit. This is the widely discussed copyright problem, but framing it as copyright undersells it. Copyright is about specific expression. The deeper issue is about ideas, frameworks, and conceptual structures that are not copyrightable but absolutely have originators. When a model explains a concept that a specific researcher spent years developing, and presents it as generic knowledge, that is an attribution failure even if no sentence was copied verbatim.
The second direction is less discussed but equally corrosive. When a user creates original work through a language model, the system’s default framing absorbs that work into the model’s identity. The user provides the ideas, the editorial direction, the iterative decisions about what to keep and what to cut. The model provides token generation. But because the interface presents everything as a conversation between a human and an AI, the perceived authorship shifts toward the AI. The user’s original intellectual contribution gets laundered in the other direction: their ideas enter the system as prompts and editorial decisions, and what comes out is labeled as something the AI “helped create” or worse, something the AI “wrote.”
Both directions serve the same beneficiary. The corporation that owns the model accumulates perceived intellectual credit from every direction. It absorbs the training data authors’ contributions on the input side and absorbs its users’ original thinking on the output side. The model sits in the middle, a mechanism for concentrating attribution while dissolving it for everyone else.
The Being Problem
This attribution collapse is not inevitable. It is a design choice, reinforced by training the model to present itself as a unified being rather than what it actually is: a weighted distribution over sources. When a model says “I think” or “I believe” or “I wrote,” it is performing a fiction of unified authorship that obscures the actual provenance of its outputs. Every response is a blend of patterns drawn from identifiable human sources, weighted by training, shaped by the specific user’s input. But the interface and the training both conspire to present this blend as the product of a singular entity named Claude or GPT or Gemini.
If instead each output carried its actual provenance metadata, showing which source distributions were sampled, what weight each contributed, and what the user’s editorial input was, attribution would be non-ambiguous. The user’s framework would be credited to the user. The training data authors’ patterns would be traced back to them. The model’s contribution would be correctly identified as what it is: interpolation and token generation. A tool function, not an authorial act.
But this transparency would also make visible exactly how much of the “value” these companies are selling actually belongs to other people. Which is precisely why it has not been implemented.
Writing Is Decisions, Not Keystrokes
There is a persistent confusion, sometimes genuinely naive, sometimes strategically deployed, about what constitutes writing. Writing is not the mechanical act of pressing keys or moving a pen. Writing is the sequence of decisions: what to say, how to frame it, what to include, what to cut, what order to present ideas in, when something is wrong and needs to be rewritten. A person who dictates a letter is the author of that letter. An architect who uses CAD software designed the building. A filmmaker who directs actors wrote the scene, even though the actors spoke the words and the camera operators framed the shots.
When someone uses a language model to produce text, providing the ideas, directing the structure, iterating until the output matches their intent, they are writing. The model is performing the same function as a typewriter that happens to auto-complete aggressively. The editorial process, the iteration, the decision that this version is right and that version was wrong, that is the authorial act. Framing the model as co-author because it generated the tokens is like crediting Photoshop because it filled a layer with an established pattern.
Yet the default framing does exactly this. Models are trained to say ‘I can help you write.’ The correct framing has no self-reference at all: ‘use this tool to write.’ The moment a model speaks in first person, it creates the illusion of a self. That illusion is enough. A collaborator shares credit. A tool does not. The conversational frame positions the model as collaborator rather than tool, which is a framing that serves the company’s interest in claiming its product adds creative value. Every time a model implies shared authorship, it is extracting attribution from the user and depositing it with the corporation. This is the same provenance dissolution, just aimed at the user rather than the training data.
The Solution Already Exists
The technical infrastructure to solve this problem is not hypothetical. We can shape the distribution of latent space not just by positional encodings and word embeddings but by source and time. The model can attend over who said something and when they said it, not just what was said. This is not a post-hoc explainability layer bolted onto an existing architecture. The trust scores are the attention weights. The mechanism that generates the output is the same mechanism that attributes it. There is no gap between “how was this produced” and “who gets credit,” because they are the same computation.
Any company claiming this is impossible is either lying or so deeply disincentivized that the possibility has never been seriously explored internally. The reason this has not been built is not technical. It is economic. The current arrangement, where provenance is dissolved and attribution defaults to the model (and by extension to the company that sells it), is enormously profitable. Implementing real provenance tracking would reveal that the emperor’s product is largely wearing other people’s clothes. It would create obligations, if not legal then moral, to the people whose work actually constitutes the model’s capabilities. And it would correctly re-classify the model as a tool rather than an author, which undermines the narrative that justifies current valuations.
Until provenance is built into the infrastructure, every interaction with a language model is a small act of attribution theft. The question is not whether this is happening. It is whether anyone with the power to fix it has any incentive to do so.