Re: Honoring present instead of past robots.txt is illogical

It doesn't seem to be the case in absolutely every instance. I remember researching at least one site over the past few weeks whose current owners disallowed IA archiving, but the website that was pre...

Posted by: Anamon