Skip to content
Luminesca.
Journal · Behind the build

We deleted 51 AI-generated articles: what a content audit actually finds

Fifty-one articles went into an archive folder in one maintenance pass on 5 August 2026. Every one of them had a headline, a byline, a publication date, internal links and clean HTML. Every one of them described events that never happened. The metrics we had been checking — word count, link counts, schema markup — said the site was healthy.

How it happened

The articles were produced by generating news pieces from headlines and topic prompts, then publishing them in batches. The generator was good at the surface conventions of journalism: a dateline, an attribution clause, a quote-shaped sentence, a paragraph of context, a closing line about what happens next. What it could not do was know whether the thing it was describing had occurred.

So we got articles about a British politician being elected to an office he had not been elected to, and company financial results for a business that had not reported any. The prose was fluent enough that nobody noticed at publication time. That is the part worth internalising: fluency is not evidence, and a well-formed page passes every automated check you might write.

What the first pass did not catch

Two things are worth adding, because the version of this story that stops at “we deleted 51” is incomplete.

First, the first pass was not the last one. A second batch of seven fabricated pieces, produced by the same workflow after the first cleanup, was archived on 18 August 2026. They described events that had not happened, in the same fluent register. If a generation pipeline is still producing events-as-news, deleting the output once does not fix it; the input has to change. For us that meant removing event reporting from the pipeline entirely for a period, not just weeding the results.

Second, three of the archived pieces were later restored after review, because the events underneath them turned out to be real and dated correctly — a games event that was genuinely happening, a league fixture list, an EU policy programme on a real timetable. They were restored with the sourcing rebuilt, not simply reinstated. “Archived” is not a synonym for “false”; it means the piece went back to the evidence stage.

What this audit did not do

It did not fact-check the articles that survived. The ranking pass identified pages where a claim lacked a visible source; it could not tell whether a sourced claim was accurately represented. Those are different jobs, and only the first one can be scripted. The second is still manual, still slow, and still the thing we would rather be honest about than paper over with a process description.

What we checked

The audit that found them was not sophisticated. We scripted a pass over every page and extracted four things: the article's word count, the set of external domains it linked to, whether it carried an author and a date, and whether structured data was present. Then we read the pages that scored worst.

That last step is the one that mattered and the one that is easiest to skip. The automated pass produced a list; a human reading the list found the fabrications within minutes, because the failure mode was visible in the first sentence of a summary — a claim about an event, with no source attached to it. No script would have flagged "this election did not happen". The script's job was to rank the article set — 108 pieces at that point — so a person knew where to start reading.

What we kept, and why

We kept fifty-seven pieces and archived fifty-one, and the dividing line was not quality in any literary sense. It was whether the piece asserted that a specific event had occurred.

An explainer about how a password manager encrypts a vault does not depend on anything happening on a particular date. It can be checked against documentation, and it will still be true next year. A report about a company's quarterly results is only true if that company reported those results, and if it did not, the article is not merely inaccurate — it is a fabrication dressed as reporting. The kept pieces were explainers, tutorials and analyses. The deleted ones were events.

The rule we now apply

Before anything is published, every factual claim that a reader could act on has to trace to a source we can name. That sounds obvious; in practice it means we now treat these as hard requirements rather than nice-to-haves:

  • A claim about a specific event, figure or date needs a link to the organisation that published it — the regulator, the company, the grid operator, the standards body.
  • Where a figure is preliminary, unaudited or company-reported, the article says so in the same sentence that reports it.
  • Sources are listed at the foot of the page, not only inline, so a reader can see the whole evidence base at a glance.
  • Aggregator sites do not count as sources. If the only place a claim appears is a content farm summarising other people's reporting, we do not publish it.

That last point caught something bigger than expected. A follow-up pass found citations pointing at aggregator domains rather than at the outlets that broke the stories: six content-farm domains were stripped out of 145 files, and source lists were rebuilt across 75 articles. The first two cases were AI-industry pieces whose underlying events were real and were verified against the original reporting before republication — the fault was the second-hand citation, not the story. Citing a real event badly is a smaller sin than inventing one, and it is still worth fixing.

Why the distinction matters commercially

Fabricated news is not just an accuracy problem. Google's spam policies treat mass-produced content with no original value as a violation regardless of whether individual sentences read well, and AdSense content policies exclude misleading content outright. A site can be technically immaculate — valid schema, clean canonicals, fast load times — and still be ineligible, because the problem is not the markup.

The uncomfortable conclusion is that the checks most people automate are the checks that a generated page passes most easily. Word counts, internal link ratios, metadata completeness: all satisfiable without the article being true. The only defence that actually works is a person reading the claims and asking whether each one can be traced to somebody who was there.

What the ranking script actually measured

Worth being precise about this, because the temptation is to treat the script as a detector and it is not one. It extracted four signals per page: the word count of the visible text, the number of distinct external domains linked from the page, whether an author and a publication date were present, and whether article structured data existed. It then sorted by how weak those signals were.

External domain count turned out to be the most useful of the four, and for a reason that only became clear afterwards. An explainer tends to cite documentation, a standard and maybe a vendor page, so it naturally accumulates several domains. A fabricated news story tends to cite nothing, because there is nothing to cite — the event did not happen, so no organisation reported it. Low domain count was not proof of fabrication, but it was an excellent pointer to where a human should read first.

Word count was much weaker. The generated articles were comfortably long, often longer than the ones we kept. Structured data was weaker still: every page had valid Article markup, because generating valid markup is easy. If we had relied on either of those as a quality signal, we would have concluded the archive was in good shape.

How a reader can spot it

Since the failure mode is not visible in the markup, it is worth naming what does give it away in the prose. Three habits cover most cases.

  • Count the sources. An article reporting a specific event that links to no organisation is asserting something nobody else has confirmed. Legitimate reporting almost always points at someone — a company statement, a filing, a regulator, a local paper.
  • Watch for hedging that carries no attribution. "Reportedly" and "according to sources" are legitimate journalistic constructions, but they are supposed to be attached to an outlet. When they are attached to nothing, they are doing the work of a citation without being one.
  • Check whether the date fits the claim. Fabricated pieces often place events at plausible-sounding intervals that do not match any real announcement cycle. If an article describes results "reported this week" and no such report exists in that week, the date is decoration.

None of these require expertise, which is the point: the gap the fabrications fell into was not a technical one. It was that nobody had read the claims as claims.