Bot View
See how 274 search engines, AI crawlers and SEO tools are treated by a page's robots.txt, llms.txt, meta robots and TDM signals.
Bot View answers "which bots can read this page?" for every known crawler at once, with the robots.txt rule and line behind each verdict, and checks the other signals sites use for crawlers and AI: llms.txt, meta robots, X-Robots-Tag, Content-Signal and TDMRep.
Pages
Check how crawlers see a pageSee which of 274 search, AI and SEO crawlers a page's robots.txt allows, with the deciding rule, plus llms.txt, meta robots and TDM signals.Start a Bot View jobCheck up to 10,000 URLs with Bot View in one background job; each item takes the same parameters as a single call.Get a Bot View jobCheck a Bot View job: its status, how many items are done and failed, and when its results expire.Get Bot View job resultsRead a Bot View job's items as each one finishes, with its full answer or error, and page through them with a cursor.Cancel a Bot View jobStop a Bot View job. Items already finished stay readable for 24 hours; items not started are dropped.