llms.txt: What It Is, Who Actually Honors It, and Whether You Should Bother

A marriage and family counseling practice approached us recently with a specific request. They had read about llms.txt files as a way to speak directly to AI bots, and they wanted one. The subtext was familiar: is this the trick? Is this the thing that gets us into AI answers?

We gave them the answer we give everyone, and it is the honest verdict this post exists to deliver: llms.txt is not a magic bullet. It is a cheap, low-risk bet that belongs inside a larger strategy, not a substitute for one. We did build one for them. We also built the machine-readable files visitors never see, tuned the meta tags that are semi-visible depending on the tag, and did the unglamorous work on content and interlinking. All of these work together; none of them works alone. The practice's AI visibility improved because of the whole stack, and the llms.txt was one modest brick in it.

What llms.txt is, in two paragraphs

llms.txt is a proposed standard: a plain markdown file at your site's root (yoursite.com/llms.txt) that acts as a curated index of your best content for AI agents. Where robots.txt tells crawlers what they may not touch, llms.txt tells AI systems what is most worth reading, with short descriptions and links, so an agent with a limited attention budget spends it on your definitive pages instead of your privacy policy.

The format is deliberately simple. A title, a summary of who you are, and organized sections listing your key pages. Any competent web person can produce one in a sitting, and it needs updating only when your cornerstone content changes.

The adoption truth

Here is the part vendors selling "AI SEO" packages tend to blur. Adoption is real among tool makers: developer platforms, documentation hosts, and a growing list of AI-adjacent companies publish and parse llms.txt files. But the major LLM providers have not committed to honoring it. There is no confirmed weight in how ChatGPT, Claude, Perplexity, or Google's AI features select sources today.

So the honest classification: llms.txt is not a ranking lever you can pull. It is optionality. If the majors adopt it tomorrow, you are already standing where the line forms. If they never do, you spent ten minutes.

Why we build them anyway

Two reasons, and the second one matters more than the file.

First, the cost-benefit is lopsided. Ten minutes of work, zero risk of harm, nonzero chance of future benefit. Cheap insurance is worth buying when it is genuinely cheap.

Second, and this is the quiet payoff: writing an llms.txt forces you to answer a question most businesses have never answered explicitly. Which pages ARE your most authoritative? If you cannot fill out a one-page index of your definitive content, the file is not the problem. That difficulty is diagnostic, telling you the shape of your coverage needs work before any indexing of it will matter. When the counseling practice went through the exercise, choosing what belonged in the file clarified their whole content plan.

The verdict

Build one. Spend ten minutes, not ten hours. And do not let anyone sell it to you as the strategy. The strategy is still what it has always been, the foundation we laid out in SEO and GEO are not rivals: deep coverage, clean structure, real markup, and links that teach machines how it all connects.

If picking your definitive pages turns out to be the hard part, that is the part worth fixing first. Topic Modeler maps what your site actually covers and how strongly, so the "best of" list writes itself and the gaps become your next assignments. Ten-minute file, honest foundation under it.