CiteFuel Research

Original data on AI crawlability configuration.

CiteFuel publishes focused research on how websites expose technical configuration signals used in search and AI-answer workflows. Each publication identifies its sample, date, measurement boundary, limitations, and available data. CiteFuel is a VantaWeb company; it is not an independent newsroom.

June 2026 · Configuration inventory

The State of AI Crawlability 2026

We crawled 10,000 popularity-ranked Tranco domains and received parseable homepages from 7,543. The study inventories five binary signals: llms.txt presence, effective robots.txt policy for five crawler or product tokens, Organization schema, a canonical tag, and an HTTPS redirect.

It did not measure rankings, recommendations, citations, traffic, or the current CiteFuel product's 26-check score.

Read the report → Aggregate JSON

Measure your site’s AI-readiness gaps, then review the evidence.

Free 26-check audit. No card. No login. Just a URL — results in ~90 seconds.

Audit my site free →