Robot Invasion Open Thread

Sun, Sep 7, 2008 at 6:33:14 pm PDT

We’re being besieged by web crawlers again, trying to download every page at the site as quickly as possible, ignoring the rules in our robots.txt file, and generally being a huge pain in the posterior. They always seem to get active on weekends, probably counting on people to be less vigilant.

Here’s an open thread while Stinky and I break out the delousing tools.