LGF Front Page > TagViewer (entries by tag)
We’re being besieged by web crawlers again, trying to download every page at the site as quickly as possible, ignoring the rules in our robots.txt file, and generally being a huge pain in the posterior. They always seem to get active on weekends, probably counting on people to be less ...
In the past few hours, I’ve discovered and blocked at least six web crawlers trying to go through the site and download everything as quickly as possible, in violation of the rules in our robots.txt file. Two of them seemed to be focusing on the posts related to the Georgia ...
A web crawler using the IP address 66.165.190.122 has been hitting us like crazy for the past couple of hours, violating the rules in our robots.txt file and crawling through LGF as quickly as possible. When I checked the access log, I discovered something that makes me a bit suspicious; ...
Thousands more warbots are being readied to hit the ground in Iraq. In December, after all the lawsuits, the private eyes and the backroom deals, iRobot finally wrestled away the biggest ground-robotics contract in military history from its former employee and his secret partner. This “unmanned surge” was worth ...
A robot from Digg.com has been rapidly running through everything at LGF, including images, with multiple hits per second. It’s doing this despite the following lines in our robots.txt file: User-agent: * Crawl-delay: 600 This rule is supposed to limit the amount of hits from all robots to no more ...
Horrible Japanese robotic violins. No sax, though. (Click picture for video.)
Evil neocon robotic insects are spying on the Kos Kidz! Daily Kos: WaPo: Robotic Insects Spy on Protestors? The next time you are at an anti-war protest and you see a dragon-fly in the corner of your eye, perhaps you should look more carefully. The Washington Post has a ...
LGF, Daily Kos, Moonbats, Useful Idiots, Progressives, Robots
Here’s an open thread for a Friday afternoon; I just caught some high-speed Japanese robots hitting our pages over and over and racking up the bandwidth. Blocked ’em. No, I don’t mean this kind of Japanese robot:
[Video] (Hat tip: George.)
Our creepy Japanese toddlerbot of the day. The Child-Robot with Biomimetic Body, or CB2, looks at Shuhei Ikemoto, an Osaka University student, when its hand is clasped by him during a demonstration of the toddler robot at a lab of the university in Osaka, western Japan, Friday, June 8, ...
I got those old spambot blues this morning. We seem to be under a major attack by referrer spammers, and that’s why the “Visitors Online” counter went so high. I’m shutting them down; please stand by. UPDATE at 12/15/06 8:44:22 am: The attack seems to be directed at quite a ...
If you’ve been experiencing a sudden surge of soul-numbing email spam in your Inbox, here’s the reason. A scary little article at eWeek looks at the huge number of compromised (i.e. virus/worm infected) servers, and what the spammers are making them do: ‘Pump-and-Dump’ Spam Surge Linked to Russian Bot Herders. ...
Israel is working on tools for waging asymmetrical warfare more effectively: Israel developing anti-militant ‘bionic hornet’. (Hat tip: LGF readers.) Also: super gloves and suicide bomb sensors. JERUSALEM (Reuters) - Israel is using nanotechnology to try to create a robot no bigger than a hornet that would be able to ...
LGF reader Earth2moonbat pointed out that if you search Google for ‘LGF’ you will now see this as the first listing: Interesting! So at least one of the bots caught by our flood control system, hitting the site more frequently than it should, was from Google. UPDATE at 11/18/06 ...
The floodgates are performing very well, and our plague of robots has subsided. It’s kind of cool; I can scan through the logs and actually spot the points at which the bots get shut down. There’s a burst of sudden activity from a proxy IP, hitting a certain page several ...
These freakin’ robots that keep hitting LGF are driving me nuts (it’s a short drive, or so I’m told). So I’ve implemented a rule-based flood control script to try to shut down these instruments of evil, and retain what’s left of my sanity. What this means in sort-of-English is that ...
You may have noticed that our visit counter in the left sidebar is showing over 400,000 page views yesterday. Uh, no, not really. It’s that high because the site has been crawling with web robots for the past couple of days. The oddest one has been using a series of ...
The Air Force notices that blogs might be a valuable intelligence source in the war against radical Islam: Blogs study may net credible information. Note to the AF web geeks soaking up their share of that contract: if your web crawlers hit LGF too frequently, you’re gonna get banned. You’re ...
Isaac Asimov is spinning in his grave: Slashdot | U.S. Army Robots Break Asimov’s First Law.