Skip to content

We checked five AI statistics we were about to publish. Three had no source.

This is not a think piece about AI hype. It is the audit log of five specific numbers we were about to put in front of buyers, and what happened when somebody was made to find the primary source for each.

Published 7 min read Sources checked 24 September 2026

Filed under AI statistics, Source verification, Marketing claims

Generated art for this note.

The short answer

Of five statistics we intended to publish, one we had credited to Gartner was Gartner’s, but a count of inquiries to its analysts, not the demand we had written, and we first got that wrong too, calling it misattributed; one was stale and had quietly stopped supporting its own argument, and three had no traceable source at all, with the largest public dataset contradicting one of them outright. Two survived. They are the two now on our site.

On this page · 10 sections
  1. 1. “A 1,445% one-year surge in demand for multi-agent systems, says Gartner”
  2. 2. “40% of enterprise apps will run task-specific AI agents by 2026, says Gartner”
  3. 3. “Desktop computer-use scores in the low 80s on OSWorld”
  4. 4. “Across 500,000+ cold emails, AI personalisation performed 25% worse than sending nothing”
  5. 5. “One human running two AI seats books 1.9× more meetings per dollar”
  6. What replaced them
  7. The method, so you can do it to us
  8. What this does not tell you
  9. Questions people ask
  10. Sources

Every AI agency page carries numbers. Ours did too. In September 2026 we made somebody find the primary source for each one before it shipped.

Two survived intact. Here is the whole audit, including the one that was on our own homepage.

Five statistics, auditedOf five statistics audited, two survived with primary sources, one was misdescribed, one was stale, and three had no traceable source, with overlap between categories.FIVE STATISTICS, AUDITED21132 · Survivedprimary source, dated1 · Misdescribedthe source said less1 · Staleand self-defeating3 · No source foundone contradictedFive claims. Seven verdicts, because two failed twice.Five statistics, auditedOf five statistics audited, two survived with primary sources, one was misdescribed, one was stale, and three had no traceable source, with overlap between categories.FIVE STATISTICS, AUDITED21132 · Survivedprimary source, dated1 · Misdescribedthe source said less1 · Staleand self-defeating3 · No source foundone contradictedFive claims. Seven verdicts, because two failedtwice.
Two of five survived contact with a search engine. Our own audit, 7 September 2026. Method below.

1. “A 1,445% one-year surge in demand for multi-agent systems, says Gartner”

Verdict on 7 September: misattributed. Removed. On 24 September: Gartner’s, but not what we said. Still removed.

This was on our homepage. When we first chased it back we found it in a press release from Belitsoft, a software development company, found no Gartner publication carrying it, and called it a vendor’s number wearing an analyst’s name.

We were wrong about that. Gartner’s own article on multiagent systems, published on 18 December 2025, says it plainly: “Gartner reports a 1,445% surge in MAS inquiries from Q1 2024 to Q2 2025.” Belitsoft had quoted Gartner correctly. Our search had failed, and we blamed the messenger.

What was wrong was our sentence. Gartner counted inquiries to its analysts, over five quarters; we had written a one-year surge in demand. That is a smaller claim dressed as a bigger one, and the second mistake, on top of it, is the one to remember: a search that finds nothing is a fact about the search.

2. “40% of enterprise apps will run task-specific AI agents by 2026, says Gartner”

Verdict: true, but misworded. Corrected and dated.

This one is genuinely Gartner’s: press release of 26 August 2025, predicting 40% by 2026, up from less than 5% in 2025. But Gartner published “feature”, and we had written “run”.

One verb. It is the difference between quoting an analyst and paraphrasing one, and only one of those is checkable. It now matches, and carries the date.

3. “Desktop computer-use scores in the low 80s on OSWorld”

Verdict: stale, and it had stopped making its own point. Replaced.

We recorded the leaderboard’s best as 86.1%. Read again on 24 September 2026, the best model shows 86.0% and no entry reads 86.1%, so we cannot reproduce our own figure; the point survives it. The human baseline on OSWorld is 72.36%, so agents now score above humans on that benchmark.

We had been using “low 80s” to imply a ceiling. At 86% against a 72.4% human baseline it implies the opposite. The number had drifted past the argument it was hired to make.

A stale statistic does not just become inaccurate. It can quietly start arguing against you, and nobody notices because the sentence still reads fine.

4. “Across 500,000+ cold emails, AI personalisation performed 25% worse than sending nothing”

Verdict: no source found. Contradicted. Struck.

We could not locate any study matching this claim. What we did find was Hunter’s State of Email Outreach 2026, drawn from 31 million emails, reporting that two custom attributes lift reply rate by 56%: 5.6% against 3.6%.

That is not a small discrepancy. It points the other way, from a dataset sixty times larger than the one our claim cited.

5. “One human running two AI seats books 1.9× more meetings per dollar”

Verdict: no source found. Struck.

Nothing. No study, no dataset, no vendor even claiming it. It had presumably been absorbed from a conference slide.

What replaced them

ClaimSourceWeight
40% of enterprise apps will feature task-specific agents by end of 2026Gartner, 26 Aug 2025Analyst, primary
86.0% best model on OSWorld-Verified, vs 72.36% human baselineOSWorld-Verified leaderboard, read 24 Sep 2026Public leaderboard, primary
31.4% of long-horizon tasks completed by the best agentOSWorld 2.0 leaderboard, read 24 Sep 2026Public leaderboard, primary
Manually edited emails beat fully automated by 18%Hunter, 31M emailsVendor dataset, large, method stated

The method, so you can do it to us

  1. Find the primary source, not the article citing it. Most “according to Gartner” claims resolve to a blog citing a blog citing a press release.
  2. Check the date. Benchmarks move. A true 2024 number can be a false 2026 one.
  3. Check the wording against the original. “Run” for “feature” is how a checkable claim becomes an unfalsifiable one.
  4. Ask who profits. A vendor dataset is not worthless (Hunter’s 31 million emails are worth more than most academic samples) but it must be labelled.
  5. Write the date you checked it, so the next person knows how stale it is without repeating the work.

Every page on our site now lists the sources behind its figures and the date they were last checked, and our build fails if a page has none, or if a source goes more than thirty days past its re-check. That is not a promise. It is a test that runs.

What this does not tell you

How reliable AI statistics are in general. Five claims is not a survey: we checked these because we were about to publish them, so three of five failing is our rate on our own drafts, not an industry rate. It may say more about where we had been reading than about the field.

“No source found” means none we could reach with reasonable effort, not proof that none exists. If you hold the primary source for any of the three, send it: we will correct the note and say what changed, which is the same promise the register makes.

Corrected 24 September 2026. The first finding here said Gartner never published the 1,445% figure. It had, on 18 December 2025, as a count of inquiries to its analysts; we found the page by re-checking our own register, which is the promise in the paragraph above applied to ourselves. The finding now says so, and the benchmark figures are the ones read that day. The entry is on the register.

Questions people actually ask

How do you check whether an AI statistic is real?

Trace it to the primary source rather than the article citing it, check the publication date against how fast the field moves, compare the wording to the original, note who profits from the claim, and record the date you checked.

Is the 1,445% multi-agent growth figure from Gartner?

Yes. Gartner’s article on multiagent systems, published 18 December 2025, reports “a 1,445% surge in MAS inquiries from Q1 2024 to Q2 2025”: inquiries to its analysts over five quarters, not demand over a year. We first wrote that no Gartner publication carried it; that was our search failing, corrected on 24 September 2026. The figure stays off our homepage, because the claim it was supporting was bigger than the one Gartner made.

Does AI personalisation improve cold email reply rates?

The largest public dataset we found (Hunter’s State of Email Outreach 2026, covering 31 million emails) reports that two custom attributes lift reply rate from 3.6% to 5.6%, a 56% improvement. It also found manually edited emails outperform fully automated ones by 18%.

Sources

  1. 01
    Gartner, Gartner Predicts 40% of Enterprise Apps Will Feature Task-Specific AI Agents by 2026, Up from Less Than 5% in 2025, 26 August 2025. gartner.com(opens in a new tab)

    Weight: analyst, primary Re-checked: once a year Last read: 24 September 2026

  2. 02
    OSWorld-Verified leaderboard, read in a browser on 24 September 2026: best model 86.0% (claude-fable-5, 1 August 2026); best agentic framework 90.2%; no entry reads the 86.1% we had recorded. Human baseline 72.36%, from the original OSWorld paper. osworld-v1.xlang.ai(opens in a new tab)

    Weight: public leaderboard, primary Re-checked: every 30 days Last read: 24 September 2026

  3. 03
    OSWorld 2.0 leaderboard, read 24 September 2026: 108 long-horizon tasks; best binary completion 31.4%. osworld-v2.xlang.ai(opens in a new tab)

    Weight: public leaderboard, primary Re-checked: every 30 days Last read: 24 September 2026

  4. 04
    TheAgentCompany, Carnegie Mellon, arXiv:2412.14161 (30% of tasks completed autonomously at publication); the benchmark’s leaderboard now shows 42.9%. arxiv.org(opens in a new tab)

    Weight: preprint, primary Re-checked: never: it cannot go out of date Last read: 24 September 2026

  5. 05
    Hunter, State of Email Outreach 2026: 31 million emails; two custom attributes 5.6% vs 3.6% reply; manually edited 5.2% vs fully automated 4.4%. hunter.io(opens in a new tab)

    Weight: vendor dataset, large, method stated Re-checked: every six months Last read: 24 September 2026

  6. 06
    Gartner, “Multiagent Systems in Enterprise AI: Efficiency, Innovation and Vendor Advantage”, published 18 December 2025, updated 14 May 2026: “Gartner reports a 1,445% surge in MAS inquiries from Q1 2024 to Q2 2025, reflecting skyrocketing interest.” The source we said did not exist. gartner.com(opens in a new tab)

    Weight: analyst, primary Re-checked: never: it cannot go out of date Last read: 24 September 2026

  7. 07
    Belitsoft, press release of 13 April 2026, “Multi-Agent Systems Surge 1,445% as Enterprises Move Beyond Single AI Agents in 2026”: “Gartner says that businesses asked about multi-agent systems 1,445% more between Q1 2024 to Q2 2025.” The page we found first and wrongly took for the origin; it credits Gartner, correctly. abnewswire.com(opens in a new tab)

    Weight: vendor press release, quoting the analyst Re-checked: never: it cannot go out of date Last read: 24 September 2026

Checked 24 September 2026. Next scheduled check 24 October 2026. Numbers that move (leaderboards, live indices) are re-checked every 30 days; annual datasets and rules in force every six months; dated research once a year. If something here has gone stale before we got to it, tell us and we will correct it and say what changed.

Free tool What is your unanswered phone actually costing you? Returns a monthly and annual figure for recoverable revenue, compared against a quote you enter.
Cite this note

Noxia, “We checked five AI statistics we were about to publish. Three had no source”, Field notes, 8 September 2026; sources checked 24 September 2026. https://www.noxia.co.uk/field-notes/we-checked-five-ai-claims

Share

If you find an unsourced number on our site, tell us.

We will either produce the source or remove the claim, and say which in the note. That is the whole positioning, and it only works if it is testable.

Read next

What we found · 20 Sept 2026 · 7 min read · 4 sources

Nuggets and fool’s gold: the figures we took off our own site.

31 figures have been removed or corrected on this site, and all are still on its register. One was an analyst’s real number that we stretched, then wrongly called a fake. One was a dentist describing his own practice. Here is how each failed.