Re: robots.txt only gives temporary removal of pages?

I love the fact, that Robots.txt does not remove everything. This saved millions of pages (incl. YouTube) from permanent loss. Archive.li has webmaster protection however.

Posted by: #Danooxt3