Wridder For organizations →
← All signals

Platform Shift

Cloudflare blocks AI crawlers by default

Since July 2025, Cloudflare’s default keeps AI crawlers out unless they pay. It says getting traffic from OpenAI is 750 times harder than from the Google of old.

On July 1, 2025, Cloudflare changed its default to block AI crawlers unless they pay for the content they take. Its reasoning: search used to send visitors back, AI answers don’t. Cloudflare says it has become almost 10 times harder for a creator to get the same traffic from Google than a decade ago, 750 times harder from OpenAI and 30,000 times harder from Anthropic. “Increasingly we aren’t consuming originals, we’re consuming derivatives.”

Why it matters for communities

The deal that kept the open web going was simple: let search engines read everything, and they send people back. AI answers broke that deal, and the fixes so far are walls and price tags. Communities produce exactly the first-hand knowledge AI wants most, and so far they have the least say in where it goes.

Cloudflare Blog Content Independence Day: no AI crawl without compensation! Cloudflare, along with a majority of the world’s leading publishers and AI companies, is changing the default to block AI crawlers unless they pay creators for content. Read the source →

CloudflareAI crawlers

More on Platform Shifts

  1. Reddit is turning community conversations into AI-powered product reviews. Who controls the value?Reddit Reviews uses AI to sum up what its communities say about some 10,000 products. Each comment still links back to the discussion it came from. Platform Shift
  2. Discourse says a solved problem matters more than a clickReddit’s login wall follows its ad model, Discourse argues. For a support forum, an AI answer that fixes the problem is a win, as long as the source keeps its credit. Platform Shift
  3. Reddit is moving toward mandatory loginsAfter 20 years of open browsing, Reddit will test a logged-in-only experience. Google’s AI answers send it fewer visitors, and logged-in users are where its money is. Platform Shift