Check AI crawler robots.txt policy and monitor public-site policy, sitemap, and llms.txt changes.
Probing this server's capabilities…