Comment

Try Amazon Prime Video Free for 30 Days and Support LGF

14
austin_blue12/26/2015 10:54:09 pm PST

re: #13 freetoken

Does anyone here use archive.org’s wayback machine to take snapshots of their website?

I’ve done it before, but now the Wayback machine is balking, stopping because of the robots.txt file supposedly.

Well, I didn’t have a robots.txt file in the root of the domain so I’m not sure why it balked. But I added a robots.txt file with the widest possible permission, to try and see if that would work.

But the wayback machine is still balking, complaining about the robots.txt file.

Frustrating.

I would strongly suggest e-mailing Mr. Peabody and asking him for technical advice. He seemed to be the ultimate guru, if I recall correctly.