TagsTagsIll-behaved robots are becoming a nuisance at LGF. I’ve discovered that web crawlers have been going through our comments pages, opening every user profile, even though there are tags that should specifically prohibit them from doing this. The crawlers are coming mostly from our pals in China, which doesn’t give ...
LGF, Technical Info, Javascript, Ajax, Internet Archive, Open Thread, China, Web Robots