Re: Why does the wayback machine pay attention to robots.txt

This is rubbish. Being afraid of legal action for perfectly legal activity is bad policy for something so essential. It would be very simplistic and do less harm and more good to handle this another. ..

Posted by: d0c5i5