Optimize your site for AI with llms.txt

Make your content understandable, accessible and ready for the age of artificial intelligence

What is llms.txt?

llms.txt is a text file in Markdown format placed in the root directory of your Web site (for example, https://yoursite.com/llms.txt).
The file, similar to the better known robots.txt and sitemap.xml files, works as a quick guide for AI, providing a structured and relevant overview of your site’s content. Although not yet an official standard, it is a proactive and innovative approach to future-proofing your site.

Think of it as a business card for AI: it immediately shows what’s important, where to find it, and why it’s useful, eliminating distractions such as menus, scripts, or graphics.

Benefits

LLMs (Large Language Models) such as ChatGPT, Gemini, Claude, or Llama cannot read entirely websites with complex structures, advertisements, or superfluous code.
llms.txt allows you to:

Show only relevant content, skipping unnecessary elements (menus, HTML, banners).
Guide AI to the right pages with links and descriptions.
Increase the probability that AIs will use your content correctly in responses to users.
Potentially improve your site’s visibility in AI-driven responses.
Overcome the “context” limitations of LLMs, as they can only read a certain amount of text at a time.

Limitations and disadvantages

Not an official standard: some LLMs may ignore it
If too restrictive, you risk reduced visibility

Which LLMs support llms.txt?

At the moment, there is no official and widely disseminated information confirming which specific LLMs actively read llms.txt files. Nevertheless, implementing this file may prove to be a forward-looking strategy in view of possible future integrations.

Logic suggests that models developed by companies that have shown interest in optimizing web content understanding for AI could be among the first to consider or implement such a mechanism.
This could potentially include models such as:

Google Gemini
OpenAI (ChatGPT)
Anthropic (Claude)
Perplexity AI

How is an llms.txt file structured?

It follows a simple Markdown structure:

# Site name
Short description (1-2 rows)
Main sections (Documentation, Blog, Products..)
Lists of links with descriptions for each section
(Optional) Extra or secondary content.

Example:

# YourSite.com
> Short description.

## 📚 Courses
- [Page title](URL): Description

## 📰 Blog
- [AI and Jobs](https://your-site.com/blog/ai-jobs): How AI is changing the professions.

llms.txt and llms-full.txt: the differences

llms.txt = index + descriptions: AI follows links to read content.
llms-full.txt = full content in a single file. More convenient, but beware: it may be too long for the context limits of LLMs.

Menu