If you've decided to charge AI crawlers instead of blocking them, the next question is how. In 2026 there are three credible answers, and they work in genuinely different ways: Cloudflare's Pay Per Crawl, content marketplaces like TollBit, and the open x402 protocol enforced at your edge. This is the honest comparison — what each does, where it fits, and the catch each one's launch announcement skips.
Cloudflare Pay Per Crawl
What it is. A feature of Cloudflare's crawler-control suite. You flip on a per-request price for your zone; a crawler that hasn't paid gets an HTTP 402 with the price; Cloudflare acts as merchant of record and handles the money. Big publishers — the kind whose names you'd recognize — have been onboarded.
Where it fits. You're already on Cloudflare, you want the simplest possible on-switch, and you're happy for Cloudflare to sit in the middle as merchant of record.
The catch. As of mid-2026 it's still gated behind a closed beta, so "turn it on today" may mean "join a waitlist." Pricing is typically a flat per-request number for the zone, which is blunt if your /research/ pages are worth far more than your /tag/ pages. And a third party is the merchant of record on your revenue. It's the least work and the least control, in that order.
Marketplaces (TollBit, and the rest)
What it is. A middleman marketplace sits between publishers and AI companies. You list your content; AI buyers pay the marketplace; the marketplace pays you. TollBit is the most-cited example, with thousands of sites signed up; others run similar models, and a few use revenue-share splits instead of flat per-fetch pricing.
Where it fits. You're a publisher who wants someone else to do deal-making and buyer relationships, and you value being part of a catalog AI buyers already shop from. The demand aggregation is the product — buyers come to one place to license many sites.
The catch. You're inside someone else's marketplace, on their terms and their fee. Your pricing power is partly the marketplace's pricing power. And you're betting on that intermediary's buyer relationships continuing to convert — if the buyers aren't there, neither is the revenue. It's a real option for content businesses; it's a layer of dependency for everyone else.
x402 at your own edge
What it is. The open x402 protocol, enforced at your edge with your rules. A crawler hits a priced path, gets a signed 402 challenge, pays in USDC, and retries with proof. You write the rules — price by path, by method, by crawler — and earnings land in your wallet automatically. No marketplace, no merchant-of-record in the middle, no waitlist.
Where it fits. You want to own the pricing and the relationship, price different content differently, and not be locked to one CDN's beta or one marketplace's catalog. It works whether you're a publisher, an API, or both — the same pay-per-call mechanism underneath.
The catch. It's a protocol, not a managed catalog, so there's no marketplace conjuring buyers for you — you're charging the crawlers that already hit you. (For content sites, the setup is still near-zero: connect Cloudflare, a Worker enforces your rules, no origin changes — the walkthrough is in charge AI crawlers for site owners.) And, like every option here, it pays in proportion to the crawler traffic you actually get.
Side by side
| Cloudflare Pay Per Crawl | Marketplace (TollBit, etc.) | x402 at your edge | |
|---|---|---|---|
| Who's in the middle | Cloudflare (merchant of record) | The marketplace | No one — you own it |
| Pricing granularity | Flat per-zone (blunt) | Catalog / negotiated | Per path, method, crawler |
| Buyer demand | Crawlers hitting you | Aggregated by marketplace | Crawlers hitting you |
| Availability (mid-2026) | Closed beta | Live, sign-up | Live, self-serve |
| Lock-in | One CDN | One marketplace | Open protocol, portable |
| Best for | Cloudflare users wanting one switch | Publishers wanting deal-making done for them | Owning price + relationship |
How to choose
- Want the absolute least work and you're on Cloudflare? Pay Per Crawl, once you're off the waitlist. Accept the flat pricing and the merchant-of-record arrangement as the price of simplicity.
- A content business that wants buyers brought to you? A marketplace earns its fee if its buyer relationships are real for your category. Read the terms on pricing power and exclusivity.
- Want to own pricing, vary it by content, and stay portable? x402 at your edge. You're not renting a catalog slot or waiting for a beta; you're charging the traffic you already have, on your terms.
These aren't mutually exclusive, either — a marketplace listing and an x402 edge rule can coexist. The mistake is doing none of them and leaving the crawl traffic free.
One honest caveat for all three
Per-request prices are small, and at low traffic the early numbers are modest across every option here — that's a property of the traffic, not the tool. Charging is the move that compounds: it scales with crawl volume, costs little to keep running, and stops the free extraction in the meantime. We argued the skeptics' side of this directly in is x402 worth it yet?.
Next steps
- The mindset shift first: don't block AI crawlers — charge them
- The edge walkthrough: charge AI crawlers (Cloudflare path)
- For site owners: charging AI crawlers for your content
- Start free, no card: create an account · pricing