Show top rated links | LinkViewer
Suspicious Robot Sighting
Tue, Jun 3, 2008 at 9:16:42 am PDT
A web crawler using the IP address 66.165.190.122 has been hitting us like crazy for the past couple of hours, violating the rules in our robots.txt file and crawling through LGF as quickly as possible.
When I checked the access log, I discovered something that makes me a bit suspicious; all of the referring pages for this bot were from “right-wing” sites, and specifically from a limited group of anti-jihad sites including My Pet Jawa, Jihad Watch, and several others. Apparently someone is making an index of those sites for some unknown reason.
I blocked the IP in our htaccess file, and I suggest that if you run a blog you do the same.
78 comments
- Comments are open and unmoderated, and do not necessarily reflect the views of Little Green Footballs.
- Obscene, abusive, silly, or annoying remarks may be deleted, but the fact that particular comments remain on the site in no way constitutes an endorsement of their views by Little Green Footballs.
- Posts that contain phone numbers, street addresses, email addresses or other personal information will also be deleted, as will posts that consist only of a variation on the word, "First!"
- Comments that advocate violence will be cause for immediate banning with no appeal.
- REMEMBER: posting comments at LGF is a privilege, not a right. Abuse that privilege, and your account will be blocked.
