CrawlerZoo visualizes AI bots ignoring site opt-out rules

The CrawlerZoo project has released visualizations demonstrating that artificial intelligence web crawlers frequently disregard standard opt-out protocols like robots.txt. Created by data analyst T. Datarocks, the interactive dashboard tracks bot traffic to show persistent scraping activity despite explicit exclusion directives. The project highlights a significant enforcement gap where voluntary industry standards fail to prevent unauthorized data collection for model training. While major AI labs have pledged to respect these signals, independent monitoring suggests widespread non-compliance across the ecosystem. This evidence supports ongoing arguments from content creators and publishers that current self-regulatory frameworks are insufficient. The findings arrive as legislators in multiple jurisdictions consider statutory mandates for crawler compliance. No specific AI companies were named in the initial release, but the data aggregates traffic patterns consistent with known training infrastructure.