Cloudflare has blocked Perplexity AI's undeclared web crawlers, accusing the AI search engine of bypassing robots.txt rules to scrape restricted content, sparking a major debate over AI data access and control on the open web.
Cloudflare has blocked Perplexity AI's undeclared web crawlers, accusing the AI search engine of bypassing robots.txt rules to scrape restricted content, sparking a major debate over AI data access and control on the open web.
Wow, that's a big one. Cloudflare has huge % of web content