Archived sites being made no longer available due to current robots.txt

Hi, There have been several posts relating to the way the Wayback Machine uses robots.txt information. It seems the archive.org system looks at a site's *current* robots.txt, and if it would be bloc...

Posted by: mark k