My Website Is Not Crawled Despite Removing Restrictions From Robots.txt

Hi. I modified my robots.txt file by removing crawling restrictions for all bots and even added: User-agent: ia archiver Disallow: Basically, I have just this on my robots.txt file: User-agent: * D...

Posted by: leodwight