According to TechCrunch, Cloudflare's new crawler rules took effect on September 15, and the default settings now block "mixed-use" crawlers from any page carrying ads. Rather than sorting bots into AI and non-AI, Cloudflare now classifies them by function — Search, Agent, or Training — so site owners can permit one and refuse another. AI companies were given until this date to run separate crawlers for each purpose or risk being blocked across many publisher sites. The new default applies automatically to new customers, new sites, and all existing free-tier users, though owners can still change it manually. The practical effect is that the burden flips: AI firms now have to declare what they are crawling for, and publishers get a veto by default.