#Keeps spiders out of certain files or folders User-agent: * Disallow: license.htm Disallow: emailus.htm Disallow: /cgibin/ Disallow: /packets/ Disallow: /logs/ Disallow: .domains Disallow: .domains.bak Disallow: .mreply.rc Disallow: .mreply.log Disallow: query.txt Disallow: mailordr.txt Disallow: query1.txt