Мировые Новости! |
Seo |
---|
Home RSS Email Stat |
---|
Seo |
Навигация |
Информационный портал ! | Информация. |
---|
|
---|
Does a disallow in robots.txt actually delete archived content?Suppose I have a web site without a robots.txt, which is archived by archive.org. A year later, it shows up in the index and I add a robots.txt disallowing archive.org's crawler. Predictably, archive...
Posted by: phdstudent Добавлено: 09 января 2011
Источник: http://archive.org/
|
---|