re: #168 Buck

I don’t know… Unless Charles missed an opening. The Robots.txt says they can crawl the posts, but not any folder past comments/

Just a note: settings in your robots.txt file are only a suggestion. Most web crawlers, out of common courtesy, obey these settings, but there is no way to enforce it. Errors in crawler programming and plain old rudeness often result in “off limits” sections of a website being crawled.

I’ve noticed this a few times at my website. Perhaps not surprisingly, a large number of offending IP addresses trace back to China and Russia, the current hubs for illicit Internet activity of all kinds.