Re: Why does the wayback machine pay attention to robots.txt

I have been looking for the Authors Guild stance on robot.txt files, I haven't found much on it though. I did come across the Internet Archive's stance on robot.txt files however. Starting January 2...

Posted by: jory2