It sucks, but I can see the reason why they just blanket block EC2 rather than trying to find any more nuanced way to identify scrapers.
If I were running scraping bots on EC2, rate-limiting me isn't a real threat since the instant I get throttled for misuse I can just destroy the instance, start up a brand new one on a different IP address, and continue as I was before.
http://news.ycombinator.com/item?id=2441535
It sucks, but I can see the reason why they just blanket block EC2 rather than trying to find any more nuanced way to identify scrapers.
If I were running scraping bots on EC2, rate-limiting me isn't a real threat since the instant I get throttled for misuse I can just destroy the instance, start up a brand new one on a different IP address, and continue as I was before.