The adoption rate statistics of two llms.txt files differ by more than a factor of two, so which one should be used to allocate work?
The adoption rate gap of llms.txt given by different organizations is very large, how should the site decide the input according to which standard? Don't look at the percentage first, look at the denominator. The readings in the few studies available that day are indeed a factor of two or even more: 10.13% according to statistics of nearly three hundred thousand domains, 9.3% by looking only at the top one thousand domains, and a placement rate of 49.2% given by a sample of over two thousand three hundred sites. All three numbers are valid, and the reasons for their validity are different.
The gap comes from the denominator rather than the conclusion
| Research scope | Sample | Placement rate reading | What can be answered |
|---|---|---|---|
| Sampling by a wide range of domain names | Nearly three hundred thousand domains | 10.13% | The extent of the entire domain pool |
| Only looking at the top-level domain | Top 1,000 / Top 10,000 | 9.3% / 8.3% | Whether the top-level sites have been followed up |
| Sampling from a specific set of sites | 2,339 sites | 49.2% | Adoption rate among this type of sites |
| By industry to see placement | The top sample is cut by industry | Travel 16.0% | Whether there are peers doing in your industry |
The third line is the easiest to misread. The sample size is only a little more than two thousand, and when the site collection is selected by the researcher, the placement rate will be significantly higher, which means 'almost half of these sites', and cannot be taken as the level of the entire network. Comparing 49.2% with 10.13% directly, the 'four times difference' actually comes from two sets of denominators, not two answers to the same thing.
Data within the same head diameter is actually more stable: The first thousand domain names are 9.3%, the first ten thousand are 8.3%, the gap is less than one percentage point, indicating that the wider the head range, the lower the placement rate, and this type of curve is suitable for seeing where the scale of your own site falls in a segment.
What each of the two studies can answer
The adoption rate answers 'whether to do it now', not 'whether it is useful after it is done'. The former requires looking at whether peers have already laid out; the latter requires another type of evidence. The study of nearly three hundred thousand domain names also provided a conclusion: neither statistical analysis nor machine learning has observed any impact of llms.txt on the frequency of domain names being cited by large models.
After placement, whether it is read, and there is a more direct set of data: A study based on the logs of more than 170,000 domain name servers shows that 97% of llms.txt files did not receive any requests that month. That is to say, whether the file is there or not, and whether it is read or not, are two different things.
By combining the three sets of readings, the decision criteria become clear: the adoption rate is used to judge whether to follow the trend, the effectiveness reading is used to judge whether it is worth investing manpower, and the log reading is used to judge whether the file placed is really effective.
In addition to the placement rate, there are other indicators that should be looked at.
Sorting work by these three indicators is more efficient than sorting by adoption rate. The first is whether it is read: check if there is a request record for this path on the server, and it is meaningless to discuss the quality of the content without a record. The second is whether the content matches: the site structure, classification, and entry in the file should be consistent with the site map, and the expired instructions file after the site is revised is more likely to mislead than no file. The third is the maintenance cost: whether it can be automatically updated with the content, determining whether it will become an outdated page in half a year.
AnQiCMS's site side pushes the cost of the first two items lower: llms.txt supports automatic generation, which is convenient for large language models to understand the structure of the site content; Sitemap is also automatically generated, and checking both together can avoid the discrepancy between the description file and the actual page. This type of task only needs to be reviewed quarterly after one-time configuration, and there is no need to invest weekly.
How should this item be arranged for investment
Based on the current reading, a reasonable sorting is to first confirm that the site map is valid, the content structure is clear, and the text paragraphs are self-contained; these three items directly affect whether they can be abstracted; then release the automatic generation of llms.txt as a one-time configuration; finally, look at the adoption rate statistics. The placement rate of the travel industry in the header is 16.0%, and the significance of following up is not to fall behind, but not to expect it to change the citation results alone if the peers have already laid out.
Common questions
There is a conflict in the adoption rate numbers, which document should be believed?Compare the sample size and selection method. The three types of samples - head domain, random domain pool, and researcher-selected sites - are not interchangeable. Choose the one that is closest in scale to your own site as a reference.
The file llms.txt has been placed but not referenced. Should it be deleted?No need to delete it, but don't expect it to work independently. First, check if it has been read in the logs, and then check if the file content is still consistent with the site structure.
How often is it reviewed?Based on the frequency of site changes. Review once when the content structure or classification is adjusted, and the rest can be checked quarterly. By coordinating with automatic generation, it can be compressed into a low-maintenance cost configuration.