AI robots visited tens of thousands of times, but no one read llms.txt; what does this ratio indicate?
AI robot visited a large number of sites but almost no one requested llms.txt, this proportion indicates that it is not a collection switch: In a ninety-day observation, AI robots visited the site more than sixty thousand times, and only eighty or so hits this file. It is evident that crawling does not depend on this file, and the deciding factor for model citation is in the text of the page itself.
This statistics measures whether it is access or adoption
The aperture needs to be clarified first. The above set of numbers measures whether the robot actively reads this document, not whether it refers to it after reading, the difference is very big: the former indicates that publishing the document will not bring additional crawling, and the latter depends on whether the page can be cut into an independently citable answer. Expecting llms.txt to be the entry point of the AI era will lead to incorrect conclusions and also make people ignore the truly effective changes. Another meaning of low hit rate is that it can be regarded as an optimization plan for all actions, with input-output close to zero.
Not reading does not mean not using
Crawling and reading are two separate chains. Retrieval robots discover pages by following links and site maps, and the model consumes the already crawled text content when generating answers; the configuration file does not affect the entry point of this chain. What really affects the probability of citation is another three things: whether the paragraph can be an independent answer, whether the conclusion is placed at the front of the page, and whether the credible signals within and outside the site are consistent. The role of this document is more like navigation assistance, telling the model that has entered the site how to read the content structure, rather than trying to be crawled.
Where should it be placed within the site?
In order, do the basic surface first, then do the configuration file. AnQiCMS automatically generates the llms.txt for the site, and also places the automatic generation of the site map, link push, and access control configuration in the same background maintenance: the former solves the pronunciation, and the latter three solve discovery and permission. It is difficult to reflect the benefits on the citation by only doing files without changing the content structure; conversely, by changing long paragraphs into conclusion-first Q&A sections, changes usually appear first in the citation hits. Sites that are already doing this can consider this step as a low-cost alignment action, with almost zero cost, but don't treat it as a source of visibility.
Three specific methods for modifying the content structure
Put the conclusion at the beginning: The first sentence under each small title should directly answer the question in the title, not start with the background. One paragraph only discusses one thing: A long paragraph that carries two arguments at the same time is not friendly to extraction, and after it is split, each paragraph has a chance to become an independent answer. Write the number and applicable conditions in the same sentence: A sentence that gives only the conclusion and does not give the scope is easy to be rewritten into an uncontrollable expression when cited, and the version with conditions is more stable. After finishing the revision, use a simple method to self-check - randomly pick a paragraph to see if it can be understood independently of the context, and only the paragraphs that can be understood independently have a chance to be cited.
Common questions
What if the hit rate is so low, should we still release this file?Yes, but its positioning is a low-cost alignment action. It does not change the crawling and will not bring ranking. It is valuable as a content structure manual, not in terms of traffic.
How can we verify what the robot actually reads?Look at the request distribution grouped by identifier in the server logs. The focus is on which pages they crawled and which pages were never visited, not on whether configuration files were crawled.
Is GEO just for configuring this file?Answer: No. GEO handles whether the content can be selected and referenced by the answer engine, the core is the extractable structure and credibility of the content, and the file is just a small part of the metadata process.