... TagViewer (entries by tag)
... TagViewer (entries by tag)
Chinese robots. The Internet is lousy with them. WASHINGTON (Reuters) - U.S. concerns about the potential for cyber-attacks on critical infrastructure extended to the American electrical power grid on Wednesday and experts pointed the finger anew at Chinese hackers, among others. U.S. Homeland Security Secretary Janet Napolitano told reporters ...
We’re being besieged by web crawlers again, trying to download every page at the site as quickly as possible, ignoring the rules in our robots.txt file, and generally being a huge pain in the posterior. They always seem to get active on weekends, probably counting on people to be less ...
In the past few hours, I’ve discovered and blocked at least six web crawlers trying to go through the site and download everything as quickly as possible, in violation of the rules in our robots.txt file. Two of them seemed to be focusing on the posts related to the Georgia ...
A web crawler using the IP address 66.165.190.122 has been hitting us like crazy for the past couple of hours, violating the rules in our robots.txt file and crawling through LGF as quickly as possible. When I checked the access log, I discovered something that makes me a bit suspicious; ...
All day I’ve been watching the online visitor count suddenly jump way up, but without the referrals that would indicate a link from a high volume site. It’s happened four or five times, and the cause in each case has been a web crawler trying to load every page at ...
LGF was lousy with web bots this morning, using “zombie” machines compromised by viruses, crawling around the site like crazy, probably looking for email addresses to add to spam lists. They’re not finding any, of course, but they’re running up our bandwidth for no reason, so Stinky Beaumont and I ...
LGF, Technical Info, Blogosphere, Traffic, Statistics, Web Crawlers, Bots
One of the nice things about having your web server logs stored in a database is that you can easily see where the traffic is coming from, on a real-time basis. For example, by running this query: SELECT ip, COUNT(*) AS count, referrer, useragent FROM `log` WHERE created ...
LGF, Technical Info, Blogosphere, Traffic, Statistics, Web Crawlers, Bots