Document360 automatically generates an llms.txt file for your knowledge base site — a Markdown-formatted text file that tells large language models (LLMs) like GPT, Claude, and Gemini what your content is about, helping AI agents accurately retrieve and cite your documentation when answering user questions.
What is llms.txt
llms.txt is an open standard that gives AI language models a structured way to understand what a website contains. It is a plain-text file written in Markdown, placed at the root of a site (for example, https://yoursite.com/llms.txt), and readable by both humans and machines.
The standard exists because LLMs struggle to navigate full websites the way search engines do. A sitemap lists every URL but provides no context. robots.txt tells crawlers what to skip but nothing about what remains. llms.txt fills the gap: it lists a site's key pages with titles, URLs, and short descriptions, giving AI models enough context to decide which pages to retrieve and how to represent the content in generated answers.
Why llms.txt matters for your knowledge base
Users increasingly turn to AI assistants — ChatGPT, Gemini, Copilot, and others — to find answers before visiting a website.
Without llms.txt |
With llms.txt |
|---|---|
| AI tools have no structured view of your knowledge base | AI tools can read your content index before retrieving pages |
| Answers may cite outdated or third-party sources instead of your docs | Answers are more likely to cite your documentation accurately |
| Article descriptions serve only search engine snippets | Article descriptions contribute to both SEO and AI discoverability |
| No visibility into what AI crawlers can see | You can audit AI crawler coverage from a single file |
When to use llms.txt
- Your users use AI assistants to find answers, and you want those assistants to cite your documentation accurately rather than produce generic responses.
- You have written high-quality article descriptions and want them to contribute to AI discoverability, not just search engine snippets.
- You are preparing for a product launch and want to ensure your documentation is discoverable by AI tools from day one.
Before you begin
- To contribute to
llms.txt, individual articles must have a meta description set in the AI & search descriptions (GEO/SEO) tab. Articles without a description are included inllms.txtby title only. To learn how to add a meta description, see Article SEO. llms.txtis only generated for public knowledge bases and public articles in mixed knowledge bases.
How llms.txt works
Document360 generates llms.txt automatically from your published articles and their GEO/SEO descriptions — you do not create or edit it manually. The file is updated whenever you publish, update, or remove articles.
The file has three parts:
- Header — the project name and a one-line tagline describing the knowledge base
- Workspace section heading — articles are grouped under the workspace name (for example,
## Document360 2.0) - Article entries — each published article appears as a linked title followed by its meta description (if one is set)
Each article entry follows this format:
- [Article title](https://yourdomain.com/docs/article-slug.md): Meta description text.
- The article URLs in
llms.txtpoint to the.mdversion of each page, not the live HTML URL. This is intentional — the.mdformat gives AI crawlers clean, structured content without navigation elements or page chrome. - Each
.mdpage also includes a Documentation Index block at the top, pointing back tollms.txt, so AI agents navigating individual pages can always discover the full content index.
Add a meta description for an article
To ensure an article appears in llms.txt with a description,
- Open the article in the knowledge base portal.
- Click the More (•••) icon in the article header and select SEO. The Article settings panel appears.
- Select the AI & search descriptions (GEO/SEO) tab.
- Enter a description in the Description field, or click Ask Eddy AI to generate one.
- Click Save.
%20-%20category%20settings.png)
The article must have at least 200 preprocessed words for Ask Eddy AI to be available. To add descriptions across multiple articles at once, go to Content tools > SEO description.
Best practices
- Write descriptions as summaries, not teasers. AI crawlers use the description to decide whether to retrieve the full article. A description like "Covers how to configure SSO using SAML 2.0 in Document360" is more useful to an LLM than "Learn about SSO in Document360."
- Prioritize high-traffic and cornerstone articles. Use Content tools > SEO description to filter articles without descriptions and address the most important ones first.
- Keep descriptions between 150 and 160 characters. This length works for both search engine snippets and
llms.txtentries. - Review AI-generated descriptions before saving. Eddy AI generates descriptions based on article content. Verify they accurately represent the article's scope and terminology before saving.
- Write descriptions that work for both SEO and GEO. The same description appears in search engine results and
llms.txt— so a well-written description improves both your search ranking snippets and AI crawler context simultaneously. - Check your documentation's AI readiness. Use Document360's Agent Score for Documentation to evaluate how well your knowledge base is optimized for AI agents.
FAQ
Does excluding an article from external search engines also remove it from llms.txt?
llms.txt?Yes. Enabling Exclude from external search engine results in GEO/SEO removes the article from both sitemap.xml and llms.txt. Both files honour the same external indexing rules.
What happens if two articles have the same meta description?
Both entries appear in llms.txt, but duplicate descriptions reduce the quality of AI retrieval. AI crawlers may conflate the two articles or deprioritize them. Use unique descriptions for every article.
Does llms.txt affect Eddy AI search inside my knowledge base?
llms.txt affect Eddy AI search inside my knowledge base?No. llms.txt is for external AI crawlers only. Eddy AI's assistive search within your knowledge base uses a separate indexing mechanism and is not affected by llms.txt.
How long does it take for a newly published article to appear in llms.txt?
llms.txt?There may be a short delay after publishing. If an article is missing, wait a few minutes and refresh the file.