What Is llms.txt & Should Your Website Have One?

2. What Is llms.txt?

Quick Answer: llms.txt is a plain markdown file you place at your site’s root. It gives large language models a short, curated guide to your most important pages, so they do not have to wade through cluttered HTML. Think of it as a clean cheat sheet for AI, and one small part of generative engine optimization.

The file was proposed in September 2024 by Jeremy Howard, co-founder of Answer.AI. The full spec lives at llmstxt.org. The problem it tries to solve is real: AI models have limited context windows, and a normal web page is full of menus, pop-ups, ads, and scripts. Stripping all that to find the useful text is slow and messy for a machine.

It fixes this by doing the curation for the model. Instead of guessing what matters on your site, the AI gets a tidy markdown list of your key pages with short descriptions. There are usually two versions:

• llms.txt — a short index. Your site name, a one-line summary, and links to your most useful pages.
• llms-full.txt — an optional, longer file that holds the actual content of those pages in one place.

That is the whole idea. It is not code, it is not a plugin, and it does not block anyone. It is a text file written for machines that increasingly read your site.

3. llms.txt vs robots.txt vs sitemap.xml

Quick Answer: They are three different jobs. robots.txt controls what crawlers may access, sitemap.xml lists every page for search engines, and llms.txt curates your best content for AI models. It does not block anything and does not replace the other two. It works alongside your schema markup for AI search as another machine-readable signal.

The biggest myth is that llms.txt is a robots.txt for AI. It is not. robots.txt is a gatekeeper that says yes or no to crawlers. llms.txt is a helpful guide that says here is the good stuff. One restricts, the other recommends. The table below makes the split clear.

So they stack, they do not compete. Keep robots.txt to manage access, keep your sitemap for search engines, and add the file if you want to hand AI models a cleaner view. Removing one to add another would be a mistake.

4. What Goes Inside an llms.txt File

Quick Answer: An llms.txt file is short and human-readable. It opens with an H1 title for your business, a one-line summary in a blockquote, then markdown sections that link to your key pages with brief notes. That clean structure is the same logic behind the wider shift from search to SEO, AEO and GEO.

You do not need to be technical to read one. The format is deliberately plain so both people and machines can follow it. A basic file for a Malaysian business might look like this:

• # Business name — a single H1 line naming your company.
• > One-line summary — a blockquote describing what you do, e.g. “KL-based dental clinic offering braces, implants and general care”.
• ## Sections with links — grouped markdown links such as Services, Pricing, and Contact, each with a short description.
• ## Optional — lower-priority links the model can skip if space is tight.

The golden rule is to keep it honest and useful. Link the pages you genuinely want an AI to understand, write clear one-line descriptions, and skip the jargon. A bloated file stuffed with every URL defeats the point, which is to make your best content easy to find.

5. Do AI Engines Actually Use llms.txt? The Honest 2026 Answer

Quick Answer: Mostly not yet, with one clear exception. Google has said publicly it does not use llms.txt, and the big chat assistants have not confirmed it changes their answers. AI coding agents, though, fetch it routinely. So it helps developer tools today, not your ranking in Google AI Overviews.

This is where most articles oversell. The honest position in 2026 is that adoption is uneven. The file is a real, well-documented proposal, but support from the engines that matter for marketing is thin. The table below shows where things actually stand.

Why the gap? Search engines like Google already crawl the whole web at huge scale, so they do not need a hand-curated file, and they treat it the way they once treated the old keywords meta tag. The clearest traction is in the agentic layer, where coding assistants pull a site’s llms.txt to load documentation fast. As more AI marketing tools start acting like agents, that use may widen, but it is a bet on the future, not a guarantee today.

6. Which Businesses Get the Most Value

Quick Answer: Software, SaaS and documentation-heavy sites get the most value today, because AI coding agents read them. Online stores with large catalogues come next. A small local service business gets the least right now, though it costs little to add. For most SMEs, winning a spot in answers like a Perplexity AI mention still depends more on content than on this file.

The value of the file tracks how often AI tools actually read your kind of site. The more your audience uses AI agents to reach your content, the more a clean file helps. Here is a rough picture for Malaysian businesses.

If you run developer docs or a big online store, it is an easy yes. If you run a single-location clinic or cafe, it will not hurt, but it is not the thing standing between you and AI visibility. Your content, structure, and reviews matter far more.

7. How to Create an llms.txt File

Quick Answer: Pick your best pages, write them up in markdown, save the file at yourdomain.com/llms.txt, then test it. It takes most small sites under an hour. If you would rather not touch files yourself, our SEO team can set it up as part of wider AI-search work.

If you decide it is worth doing, the process is short. Here is the order that works.

How to set up llms.txt for your website

1. Map your best pages. List the 10 to 20 pages an AI should know about, such as services, pricing, key guides, and contact.
2. Write the file in markdown. Start with an H1 of your business name, add a one-line blockquote summary, then group the links under clear section headings.
3. Add an optional llms-full.txt. If you want the full text in one place, create a second file holding the content of those pages.
4. Save it at the root. Upload the file so it loads at yourdomain.com/llms.txt, the same way robots.txt sits at the root.
5. Test it. Open the file in a browser to confirm it loads, then paste it into an AI tool and ask questions to check it reads cleanly.
6. Keep it updated. Refresh the file when you add or change important pages, so it never points to dead links.

WordPress users have it easier still. Several plugins generate and update the file for you, so you set it once and let it refresh as you publish. Either way, review the file before you trust it.

8. Effort vs Payoff: Where llms.txt Fits

Quick Answer: llms.txt is low effort but, for now, low to medium payoff. Clear content, schema markup, and off-site mentions return far more for your time. Do those first, then add llms.txt as a quick finishing touch rather than a starting point in your SEO plan.

The smart question is not “is llms.txt good?” but “what should I do first?” Time is your scarce resource, so it helps to rank the AI-visibility tasks by what they return today. Here is how we sequence them.

It sits at number four for a reason. It is cheap, so there is little downside, but the first three tasks are what actually get you cited in ChatGPT and Google AI Overviews today. Add the file once the foundations are solid, not before.

9. Common llms.txt Mistakes to Avoid

Quick Answer: The big mistakes are treating llms.txt as a magic ranking fix, stuffing it with every URL, letting it go stale, and using it to replace robots.txt or your sitemap. Avoid those and the file does its small job well, without distracting from the work that still decides whether SEO is dead or just changing.

Most problems come from over-expectation. Keep these in mind:

• Expecting instant rankings. It is not a Google ranking factor in 2026, so do not measure it like one.
• Dumping every URL in. A file with hundreds of links is noise. Curate your best pages and cut the rest.
• Letting it rot. Stale links to deleted pages make your site look neglected. Refresh it when your content changes.
• Treating it as a blocker. It does not stop scraping or replace robots.txt. Keep your access rules where they belong.
• Skipping the basics for it. A perfect file on a thin, slow site helps no one. Fix the foundations first.

Done sensibly, it is a tidy, low-risk addition. Treated as a shortcut, it becomes a distraction from the work that genuinely earns AI visibility.

10. Conclusion

So, should your website have an llms.txt file? If you run software docs or a large online store, yes, it is a sensible, low-cost step that AI tools already read. For most other Malaysian SMEs, it is a nice-to-have you add after the work that matters: clear content, solid structure, schema, and genuine off-site mentions.

The trap is treating a small file as a silver bullet. AI visibility in 2026 is won by being the clearest, most trusted source on the questions your customers ask, not by one text file at your root. Get the foundations right, add it as a finishing touch, and you are ahead of the businesses still arguing about whether it matters at all.

11. Frequently Asked Questions

1. Is llms.txt the same as robots.txt?

No. robots.txt controls what crawlers are allowed to access on your site, using allow and disallow rules. llms.txt does the opposite job: it recommends your best content to AI models in a clean markdown format. It does not block anything, and it does not replace robots.txt. Most sites should keep both, since they solve different problems.

2. Does llms.txt help my Google ranking?

Not in 2026. Google has said publicly that it does not use llms.txt, and it has compared the idea to the old keywords meta tag that search engines ignore. So adding the file will not lift your position in Google results or AI Overviews today. Its current value is mainly for AI coding agents and tools that read it directly.

3. Do I need both llms.txt and llms-full.txt?

No, llms-full.txt is optional. The basic llms.txt is a short index of your key pages with descriptions. llms-full.txt goes further by holding the full text of those pages in one file, which suits documentation-heavy sites. A small business website is usually fine with just the short llms.txt, kept clean and current.

4. Where do I put the llms.txt file?

At the root of your domain, so it loads at yourdomain.com/llms.txt, the same place robots.txt sits. Upload it via your hosting file manager, your CMS, or a plugin if you use WordPress. After uploading, open the URL in a browser to confirm it loads as plain text before you rely on it.

5. Is it worth adding llms.txt in 2026 for a Malaysian SME?

It depends on your site. For developer docs or a large online store, yes, it is a quick, low-risk win. For a typical local service business, it will not hurt but will not move the needle much yet. Either way, prioritise clear content, schema, and off-site mentions first, then add llms.txt as a finishing touch.