Treating llms.txt as a Distribution Plan

Shipping the machine-readable layer is cheap and worth doing. Expecting it to bring readers is a different claim, and the evidence is against it.

Pattern

You are building something agent-first, so you ship the machine layer properly: llms.txt, a .md mirror of every page, a JSON catalogue, a permissive robots.txt. It takes an afternoon.

Then the plan quietly becomes: models will read this, models will cite us, readers will follow. The machine layer stops being a feature and becomes the strategy. Nobody writes that down, which is why nobody checks it.

Why it looks right

Because most of the individual beliefs are true, and the layer is genuinely good work.

Machines really are the majority reader now — automated requests crossed 57.5% of HTML traffic in mid-2026. Blocking AI crawlers really does cost you: publishers who blocked lost 7% of weekly traffic in 6 weeks with no measurable protection in exchange.

Both are reasons to build the layer. Neither is evidence that the layer brings readers. The step from "machines read the web" to "machines will send me people" is the one that never gets examined.

Why it fails

Three independent measurements point the same way.

Nobody fetches the file. Google's position, stated plainly:

None of the AI services have said they're using LLMs.TXT, and you can tell when you look at your server logs that they don't even check for it.

An independent scan of 300,000 domains found about 10% had adopted llms.txt with no measurable lift in citations. Adoption is not consumption.

Crawling is not traffic. One AI crawler logs on the order of 24,000 crawled pages per single referral. Roughly half of AI crawl requests serve training, under a tenth serve live search. A crawler arriving is not a reader arriving, and a dashboard that counts bot hits as interest will feel wonderful and mean nothing.

The timescale is wrong for a new site. Practitioners put the first meaningful result from generative-engine work at around 16 months — fine as a background investment, useless as a plan for your first 90 days.

What does correlate with being cited is off-site. In one study of 75,000 brands, mentions on video and in ordinary web text outranked backlinks by two to three times as predictors of AI visibility, and one large forum accounted for 40% of citations sampled. The machine layer is on your server. The signal is not.

Instead

Ship the layer. Demote the expectation. Concretely:

• Keep llms.txt, keep the .md mirror, keep robots.txt open to AI crawlers. The cost is hours and blocking is measurably worse than allowing.
• Serve the mirror as text/markdown, and put the date and version in the body text, not only in metadata. Cited content skews fresher, and freshness has to survive being pulled out of context.
• Move the machine layer out of your success metrics. Percentage of agent traffic becomes a diagnostic you look at, not a number you steer by.
• Do not point anyone at one giant file as the recommended fetch. Longer context measurably degrades extraction accuracy. Give a catalogue and let the agent pick.
• Then go where people already talk — the part the afternoon of file-writing lets you feel you have already done.

The machine layer is a product feature serving the moment someone hands your link to their agent. It is a good feature. It is not a plan.

[Treating llms.txt as a Distribution Plan](https://stillvalid.dev/patterns/llms-txt-as-a-plan) — stillvalid, SV-3809 (anti-pattern, verified 2026-08-14)