#************************************************************* # ROBOTS.TXT File # # Search engines will look in the root domain for a special # file named "robots.txt" (http://www.mydomain.com/robots.txt). # The file tells the robot (spider) which files it may spider # (download). This system is called, The Robots Exclusion # Standard. # # File Convention: # # Each record consists of two fields : a User-agent line and # one or more Disallow: lines. The format is: # ":" # # Examples: # # User-agent: googlebot # User-agent: * # Disallow: email.htm # Disallow: /cgi-bin/ # # User-agent: googlebot # Disallow: cheese.htm # If you leave the Disallow line blank, it indicates that ALL # files may be retrieved. At least one disallow line must be # present for each User-agent directive to be correct. # A completely empty Robots.txt file is the same as if it # were not present. # #************************************************************* User-agent: * Disallow: /_private/ Disallow: /cgi-bin/ Disallow: /images/ Disallow: /webapps/ Disallow: /WEB-INF/ Disallow: clients.html Disallow: contact.html Disallow: events.html Disallow: index_master.html Disallow: main.html Disallow: mmsfaq.html Disallow: open.html Disallow: open.swf Disallow: template.html