Technical SEO
Generate an llms.txt file for AI crawlers
Describe your site and key pages, then copy a ready to publish llms.txt file, the emerging convention for telling AI engines what your site is and what matters on it.
Key pages
# Your site name
> A one line summary of what your site is and who it is for.
Save this as llms.txt and publish it at the root of your domain, next to robots.txt.
What llms.txt is, and why AI crawlers read it
llms.txt is a plain markdown file served from the root of your domain, at /llms.txt. It gives an AI engine a short description of what your site is and which pages hold the answers, written by you, instead of leaving the engine to infer all of it from your navigation, a cookie banner, and a page of client-side JavaScript. Jeremy Howard proposed the convention in September 2024, and documentation-heavy companies picked it up first.
The file solves context, not access. Your robots.txt decides whether a crawler may fetch a page at all. llms.txt tells the engine what it is looking at once the fetch succeeds. A model summarising your homepage sees a headline, a row of feature cards, and a pricing table, then guesses what you sell. A model reading two lines you wrote on purpose does not have to guess.
No engine has published a commitment to read the file, so treating it as a ranking input would be wrong. What it gives you is a canonical description of your own product sitting somewhere a crawler can find it, which matters when the alternative is an engine reconstructing your positioning from a competitor's comparison page.
What a finished llms.txt looks like
This is the output the form above produces. The structure is fixed: an H1 with the site name, a blockquote holding the one line summary, an optional paragraph of context, then a linked list of the pages that matter.
# Rankfastt
> AI visibility tool that tracks whether ChatGPT, Perplexity, and Google AI Mode cite your brand.
Rankfastt runs your buyer prompts against the AI answer engines every day, records which brands get named, and drafts the page changes that get you cited.
## Key pages
- [How to get cited by AI](https://www.rankfastt.com/how-to-get-cited-by-ai): The full method, start to finish.
- [Pricing](https://www.rankfastt.com/#pricing): Plans, limits, and what each tier tracks.
- [Free tools](https://www.rankfastt.com/tools): One-off checks with no signup.
Save it as llms.txt and serve it from the root of your domain, beside robots.txt.
llms.txt questions
Does llms.txt affect whether ChatGPT or Perplexity cite you?
Not on its own. No major engine has published support for the file, and any tool promising citations from a text file is overselling it. What llms.txt changes is the description an engine works from when it does read your site. Citation is still decided by whether your page answers the question better than the page beside it, and by how many independent sources say the same thing about you.
Where does the file go?
At https://yourdomain.com/llms.txt, the root of the domain, beside robots.txt. Serve it as text/plain or text/markdown. A file sitting in a subfolder can be linked, but it is far less likely to be found.
Is llms.txt a replacement for robots.txt?
No, and the two do opposite jobs. robots.txt is permission: it says which crawlers may fetch which paths. llms.txt is description: it assumes the crawler is allowed in and tells it what matters. Blocking GPTBot in robots.txt while publishing a detailed llms.txt leaves you invisible either way.
How often should it be updated?
When your positioning changes or a key page moves. It is a hand-maintained file, and a stale one pointing at 404s is worse than having none. Check it whenever you would check your sitemap.
Previews the Technical Agent
An llms.txt file is one piece of AI readiness. Rankfastt keeps your whole technical footprint clean and monitors it so nothing quietly breaks.