Robot Invasion Open Thread

Charles Johnsonfollow me on twitter
Sun Sep 7, 2008 at 6:33 pm PDT • Views: 187

We’re being besieged by web crawlers again, trying to download every page at the site as quickly as possible, ignoring the rules in our robots.txt file, and generally being a huge pain in the posterior. They always seem to get active on weekends, probably counting on people to be less vigilant.

Here’s an open thread while Stinky and I break out the delousing tools.

22
Advertisement

269 comments

^ back to top ^

Name:

Pass:

Register Forgot Your Password? Account Settings Re-send Confirmation (To log in, cookies must be enabled in your browser!)

Turn off ads by subscribing!
For about 33 cents a day, our subscription option turns off all advertisements at LGF!
Read more...


► LGF Headlines

  • Loading...

► Tweeted Articles

  • Loading...

► Tweeted Pages

  • Loading...

► Top 10 Comments

  • Loading...

► Bottom Comments

  • Loading...

► Recent Comments

  • Loading...

► Tools/Info

► LGF Hits

► Resources

► Never Forget

► Statistics

► Tag Cloud

► Contact

You must have Javascript enabled to use the contact form.
 Your email:

Subject:

Message:


Messages may be published in our weblog, unless you request otherwise.
Tech Note:
Using the Contact Form
lgf cookbook 2
Reality Bites!
LGF Cookbook Volume 2

Partners
888-casino.eu Addiction Treatment Alcohol Treatment Alternative Medicine Dry Erase Board, White Boards, Whiteboards Fat Burners Front Row King Tickets Fundraisers Online Casino Sportsbook Tempurpedic Mattress Comparison Advertise here
More Partners

Compare Electricity Prices in your area. Texas Electricity is deregulated; you have the right to choose Texas Electric Rates from among many Texas Electric Companies.

Frogmarching into the future.

TwitterFacebook
LGF Pages
Recent Pages

MichaelJ
Daring Fireball: Walter Isaacson's 'Steve Jobs'
54 seconds ago
Views: 7 • Comments: 0
Tweets: 0 • Rating: 0

MichaelJ
Consumers Cooler to Amazon Prime Than Thought, Report Says
8 minutes ago
Views: 18 • Comments: 0
Tweets: 0 • Rating: 0

Turnabout is Fair Play
Chavez Allies Attack New Opponent Capriles as Jewish, Gay
24 minutes ago
Views: 18 • Comments: 0
Tweets: 0 • Rating: 0

Bob Dillon
Radiation Generates Cancer Stem Cells From Less Aggressive Breast Cancer Cells
1 hour, 23 minutes ago
Views: 35 • Comments: 0
Tweets: 0 • Rating: 0

Bob Dillon
Mossad Bolsters Activity in Tunisia
1 hour, 43 minutes ago
Views: 48 • Comments: 0
Tweets: 0 • Rating: 0

Bob Dillon
CA Prison Officers Spent More Than $1 Million on Political Advice
1 hour, 53 minutes ago
Views: 35 • Comments: 0
Tweets: 0 • Rating: 1

Bob Dillon
China Withdraws Support to Assad
2 hours ago
Views: 51 • Comments: 0
Tweets: 0 • Rating: 0

Bob Dillon
'Assad Using Chemical Warfare in Homs Attack
2 hours, 3 minutes ago
Views: 42 • Comments: 0
Tweets: 0 • Rating: 0

Bob Dillon
Weapons: Neutering Ammonium Nitrate
2 hours, 13 minutes ago
Views: 45 • Comments: 0
Tweets: 0 • Rating: 0

Randall Gross
Stabenow Up 14 on Hoekstra as Ad Backfires - Public Policy Polling
2 hours, 50 minutes ago
Views: 96 • Comments: 0
Tweets: 0 • Rating: 0

 Frank says:

I think (Abbey Road is) the best engineered, best mastered rock and roll album ever produced...except that I take exception to stereo placement. -- From "Frank Zappa talks about Faves, Raves, and composers in their g raves" - some English publication, I think. (2 & 3 from the book, "The Lives and Times of Frank Zappa and the Mothers", some Brit thing again (got it at Blue Meanie Imports in San Diego fifteen years ago.)