# See http://web.nexor.co.uk/mak/doc/robots/norobots.html for a description # of what this file is, and it's syntax. - CAD 4/30/95 # # Generally, it's: # # User-agent: agent-name # Disallow: directory on the server you wish to not have robots in # # To keep all robot's out of directory "blah", use: # # User-agent: * # Disallow: /blah # # CAD 4/30/95 I think that robots should stay out of the cgi-bin directory: User-agent: * Disallow: /7439_service_guide/ Disallow: /menu.htm Disallow: /cgi/ Disallow: /2002/ Disallow: /dev/ Disallow: /download/ Disallow: /internal_docs/ Disallow: /sp/ Disallow: /resources/ Disallow: /marketing/ Disallow: /25years/ Disallow: /flooded/ Disallow: /design/ Disallow: /confidential/ Disallow: /spanish/contact/ Disallow: /products/BTU_calc/ Disallow: /5ervice_guide/