About 10% of the sites that use llms.txt, what problem does this file solve?
The site ratio using llms.txt is very low, what problem does this file solve? A third-party check covered nearly 300,000 domain names, and only 10.13% of the sites had this file, another conclusion given by the same study is worth noting: it does not affect how the AI system views or references content. The low adoption rate and unverified effectiveness, when combined, are easy to judge as 'useless', but the design goal of this file was never to empower or ensure references; it addresses the reading path problem after the model enters the site.
It originally aimed to solve the traversal cost
The model proxy needs to find content on an unfamiliar site, and the default action is to traverse on its own: grab the homepage, grab the list, and judge layer by layer which article is relevant to the issue. The overview file writes this path in advance—a site description, a summary, an entry list grouped by topic. For sites with a large amount of content, this level of organization can indeed reduce ineffective crawling; for small sites, the pages themselves are not many layers, the cost of traversal is almost non-existent, and the low adoption rate is reasonable.
It should be distinguished that it does not control access permissions, nor does it change the other party's crawling quota. Whether it can be crawled is decided by the robots.txt layer, and the overview file is only responsible for where to read after being allowed in.
Why can't you expect it to improve the citation?
The judgment of whether to quote or not occurs at the content level. The paragraphs that can be extracted, the data and time that can be checked, and the section titles that match the semantic meaning of the problem are the input conditions for synthesizing the answer. A catalog-level file does not change the usability of the article itself, and is also unlikely to change the result of the citation alone. The difference between research and site practice is here: treating llms.txt as the whole of GEO work, and treating it as a link in the entire process of data collection and content organization, are two completely different ways of investment.
If the goal of the site is to be referenced, it is recommended to arrange the input sequence in this way: first, clarify the crawling license and list, then change the text into independent extractable paragraphs, and finally organize the overview files and entry. The first two determine whether it can be read and what part can be used, and the third determines the efficiency of reading.
How to implement it within the site
AnQiCMS side does not require manual word-by-word maintenance of this file: llms.txt supports automatic generation, making it easier for large language models to understand and index the site content; it, along with robots configuration, sitemap, and link push, constitutes the background GEO and SEO module set, covering four segments of work such as licensing, checklist, overview, and push. The only two things that require manual judgment are whether the key columns are all left with entries in the overview, and whether multi-language or multi-brand sites are separately configured.
Common questions
Only 10.13% of the sites are in use, should we follow?The basis for judgment is not the proportion, but the content volume and structure of your own site. Sites with more page levels and complex columns benefit more明显.
Do I need robots.txt after placing the file?Yes, they manage different layers. When the permission layer is missing, the overview file cannot be read.
How much traffic can it bring?It cannot be attributed to this file. A more reliable way to observe is to fix a batch of questions and record whether your site appears in the answers monthly.
Do you need to maintain it manually?It is not necessary to maintain word by word, the site can automatically generate an overview file, and human staff is responsible for checking whether the entry is consistent with the actual column.