# robots.txt - a full crawl block with one narrow exception. # Blocking a URL here does not remove it from an index: a blocked page can still # be listed from inbound links. Use a noindex header or meta tag for that. User-agent: Googlebot-Image Allow: /media/ Disallow: / User-agent: * Disallow: /