Yeah that was actually what I was thinking of for the idea, but the problem with their firewall is that it doesn’t penalize the big crawlers, like Google’s bot, because Google still sends traffic. Even if Google is sending less and less traffic over time.

That just gives the biggest companies the prize of being able to crawl all the web for free, while they offer less over time in exchange.