https://www.noxia.co.uk/field-notes/nuggets-and-fools-gold · printed from noxia.co.uk · sources checked 24 September 2026
Field note · What we found
Nuggets and fool’s gold: the figures we took off our own site.
Everyone can tell you to check your sources. Almost nobody shows you a figure that failed, because the figures that fail are usually your own. These are ours — struck from this site and still listed on its register, with the specific way each one broke.
- What we found
- Sourcing
- Statistics
- Verification
- Buying AI
The short answer
28 figures have been removed or corrected on this site, all still on its register. They failed in four ways: the real source, stretched — a genuine figure made to say more than it does; anecdote promoted to statistic — one person’s experience quoted as a rate; no source at all; and a range so wide it is not a number, every value from someone selling the remedy. And one lesson about checking: a search that finds nothing is a fact about the search.
On this page · 9 sections
This site publishes a register of every figure it uses, and of every figure it removed. That second list is the useful one, and it is the one nobody else keeps.
28 figures have been struck or corrected so far, and the register lists every one. The ones that failed on their sourcing did not fail randomly. They failed in four recognisable ways, and once you can name the four you can test almost any number you are shown.
One: the real source, stretched
The one we were proudest of catching was a 1,445% one-year surge in demand for multi-agent systems, credited to Gartner, on our own homepage. We traced it to a press release by Belitsoft, a software development company, could find no Gartner publication carrying it, and wrote it up here as the worst failure on the list: a vendor’s number wearing an analyst’s name.
It was not. Re-checking our own register on 24 September 2026, we found the Gartner article, published on 18 December 2025: “Gartner reports a 1,445% surge in MAS inquiries from Q1 2024 to Q2 2025.” The name was right. What was wrong was the sentence: inquiries to Gartner’s analysts had become demand, and five quarters had become a year. Belitsoft had quoted it correctly; we had blamed the messenger because our search came back empty.
That makes it the commonest failure on the register rather than the rarest. The source is real, and the sentence quoting it is bigger than the source: a base widened, a population swapped, a period stretched. Almost every correction we have made since is this kind.
Two: one person, promoted
A widely repeated figure held that one in seven dental appointments is a no-show. It traces to a single NHS dentist describing his own practice over one year, in a trade interview. No sample. No method. Not a British Dental Association study.
That is a perfectly good anecdote and we would happily quote it as one. It is not a rate. The same interview is also the origin of a £56,000 annual cost for that one practice — again true of him, again not a national figure.
The tell is grammatical. An anecdote that has been promoted loses its owner: "one dentist found" becomes "one in seven appointments". If a statistic has no subject, go and find who it originally belonged to.
Three: nothing at the end of the trail
Three of them simply have no source. £39 per missed UK appointment: none found. 40% of bookings made after hours: none found. £200,000 a month in salon no-show cost: none found. A fourth, 11 hours a week lost to admin by letting agents, was struck the same way — the nearest real source, Goodlord's industry survey, says only that one in five letting agents felt they were spending too much time on admin, which is a sentiment rather than an hours figure.
These circulate because each repetition looks like corroboration. Five blog posts citing each other is one unsourced claim with four echoes, and the only way to tell is to follow every link to the end. Most of the time the end is a slide.
Four: a range that is not a number
The subtlest failure. We wanted a figure for missed calls in the trades — one in five was the claim. What we found was that the published range for unanswered UK business calls runs from roughly half to 94%, and that every source was a company selling call answering. Only one stated a method, and it measured evening calls to sales and letting agencies, not the trades.
A spread that wide is not a measurement with uncertainty. It is a marketing range, and picking a value from it — even a conservative one — launders a sales claim into a fact. So we did not pick one. We built a calculator that carries none of those figures instead, where you enter your own number and it can tell you to buy nothing.
What we did with the two that could be saved
Striking a figure is not the only move. Two were replaced with something stronger.
The claim that deposits cut no-shows by 55–65% had no source. But two randomised controlled trials, published in PLOS ONE, tested a cheaper intervention on hospital outpatients, and in the first, 10,111 patients, stating the appointment's specific cost in the reminder message cut did-not-attend rates from 11.1% to 8.4%. That is a better finding than the one we lost, it is properly sourced, and it argues against buying software rather than for it. It became a note of its own.
What this costs, and why we do it anyway
Every struck figure is an argument that got weaker. The dental page we intended to write lost all three of its numbers. The trades page lost both of its. That is a real cost and we are not going to pretend it was painless.
But a firm that publishes the figures it removed is making a checkable claim about the ones it kept, and a firm that publishes only its best numbers is making no claim at all. Every remaining figure on this site names its source, its date and how much weight it carries, and the whole register is public at what we checked. If one has gone stale before we got to it, the page says so and asks you to tell us.
Apply the four questions to the next AI vendor deck you are shown. In our experience roughly half the numbers in one will not survive the first question, and the ones that fail are almost always the ones doing the persuading. We ran that exercise on five published AI claims, and on forty-five citations, with much the same result.
What this does not tell you
Whether a well-sourced figure is still right. The four tests catch stretched sources, promoted anecdotes, missing sources and marketing ranges; they do not catch a source that has been overtaken. The 12–20% on the register described a genuine benchmark and simply had its number wrong — the published figure is 30% — and every one of the four tests would have passed it, because the source was real.
On 24 September 2026 we opened every source this site had cited without a link and read each against the sentence citing it. Five of our sentences had the document right and its scope wrong.
A figure for sales and letting agencies was given as the figure for every firm the study rang. A US survey was quoted as if it covered every buyer. A vendor’s numbers for a typical secured loan case were stretched to every case, and given a country the report never names. An average agency was called a typical one. The fifth ran the other way: we had told readers that a well-documented study stated no method. Each is corrected where it appeared and listed on the register.
The same day we re-read every linked source against its sentence, and then looked for anything newer than each. That found the other kind of error: sources that were right when we read them and had since been overtaken — a consultation answered eight months before we wrote about it, a start date that had already passed, a leaderboard entry we could not find again. Those are on the register too, and they are why every source here carries the date it is next re-read.
Nor is the list complete for anyone else. The taxonomy comes from the figures that failed on our own site, so it describes our mistakes; another site’s would add modes we never made.
Questions people actually ask
How can you tell if a statistic is reliable?
Ask four questions. Who published it, in what document? How many things were measured, and by what method? Can you reach the original in two clicks? And does the source sell the remedy the figure argues for? Most bad figures fail the first or the last.
What is the commonest way a real statistic goes wrong?
It gets bigger in the retelling. The source is real; the sentence quoting it widens the base, swaps the population or stretches the period. Gartner’s “1,445% surge in MAS inquiries from Q1 2024 to Q2 2025” became, on our own homepage, a one-year surge in demand — and we then compounded it by calling the figure misattributed when our search failed to find Gartner’s page. Most corrections on our register are some version of the first mistake.
Why does a very wide range of published figures matter?
Because it usually means nobody measured. Published claims for unanswered UK business calls run from roughly half to 94%, every one from a company selling call answering, and the only one with a stated method measured a different industry at a different hour. Picking a value from a spread like that turns a sales claim into an apparent fact; the honest response is to refuse to pick and let the reader supply their own number.
What happens when a figure fails verification?
It is struck and recorded as struck. On this site 28 figures have been removed or corrected, and every one stays visible in the public register with the reason. Two were replaced by better-sourced findings that pointed the opposite way — which is the useful outcome, because a figure failing often means the argument was aimed at the wrong thing.
Sources
- Noxia’s own figures register, FIGURES.md in this repository, rendered publicly at /what-we-checked. 28 figures removed or corrected, counted by the build from the register itself. Named here: the 1,445% multi-agent surge, Gartner’s count of inquiries over five quarters, which we wrote as a year of demand and then wrongly called misattributed; “1 in 7” dental no-shows, traced to one NHS dentist describing his own practice in a trade interview; “£39 per missed appointment”, no source found; “40% of bookings after hours”, no source found; “£200,000 a month” salon no-show cost, no source found; “11 hours a week” lost to admin by letting agents, nearest real source Goodlord reporting only that 1 in 5 letting agents said they spent too much time on admin; “1 in 5 calls missed” in the trades, against a published range of roughly half to 94% with every source selling call answering. noxia.co.uk ↗ — primary, and first-party: this is our own record of our own failures, and every entry is publicly checkable on the site
- The replacement finding: in the first of two randomised controlled trials published in PLOS ONE, 10,111 hospital outpatients, including the specific cost of the appointment in a reminder message reduced did-not-attend rates from 11.1% to 8.4%. noxia.co.uk ↗ — primary; peer-reviewed randomised trials, and the reason the unsourced deposit claim was not simply deleted re-checked yearly
- Gartner, “Multiagent Systems in Enterprise AI: Efficiency, Innovation and Vendor Advantage”, published 18 December 2025: “Gartner reports a 1,445% surge in MAS inquiries from Q1 2024 to Q2 2025, reflecting skyrocketing interest.” gartner.com ↗ — analyst, primary — the page our first check failed to find
- The four-way taxonomy is ours. It is derived from the struck figures that failed on their sourcing, and nothing else, so it is a description of how our figures failed rather than a general theory of how figures fail. — our own argument, labelled as such, with its sample size stated
Checked 24 September 2026. Next scheduled check 24 September 2027. Numbers that move — leaderboards, live indices — are re-checked every 30 days; annual datasets and rules in force every six months; dated research once a year. If something here has gone stale before we got to it, tell us and we will correct it and say what changed.
Cite this note
Noxia, “Nuggets and fool’s gold: the figures we took off our own site”, Field notes, 20 September 2026; sources checked 24 September 2026. https://www.noxia.co.uk/field-notes/nuggets-and-fools-gold
Send us the deck. We will tell you which numbers survive.
Four questions, applied line by line to whatever an AI vendor has put in front of you, with the trail written down for each figure — who published it, when, on what sample, and whether they sell the answer. You get a marked-up copy. It is the same method we ran on ourselves, and so far it has found 28.
Talk to us about thisRead next
What we found
We checked five AI statistics we were about to publish. Three had no source.
One was on our own homepage, credited to Gartner. We first said Gartner never published it. Gartner had — as something smaller than we claimed. What happened when we looked, twice.
Appointments
The cheapest fix for missed appointments is one line in a text message.
Two randomised trials at a London NHS trust found that adding the cost of the appointment to the reminder text cut no-shows by roughly a quarter. Nothing was installed.
Advice & compliance
18 of 45 citations did not exist. What a court said about AI drafting.
Two cases, one Divisional Court, £2,000 in wasted costs, four referrals to regulators — and one sentence about professional duty that applies to every firm that lets a machine draft.