# --------------------------------------------------------> # Robot Exclusion File -- robots.txt # Weingarten Enterprises -- http://www.weingarten.net # Author: M. Weingarten # Website URL: http://www.weingarten.net/ # File Name: http://www.weingarten.net/robots.txt # Webhost: Econoservers.com # Last Updated: Tuesday, June 02, 2009 # --------------------------------------------------------> # Ordered as to avoid interpretation problems with bots. # Disallowed Robots --------------------------------------> # NameProtect Inc. # http://www.nameprotect.com/botinfo.html User-agent: NPBot Disallow: / # Picsearch # http://www.picsearch.com/bot.html User-agent: psbot Disallow: / # Allowed Robots / Disallowed Paths ----------------------> User-agent: * Disallow: /cgi-bin/ Disallow: /cp/ Disallow: /old/ Disallow: /archive/ Disallow: /wp-*/ Disallow: /images/ Disallow: /rssjs/ Disallow: /js/ Disallow: /css/ Disallow: /demo/ Disallow: /private/ Disallow: /uploads/ Disallow: /downloads/ Disallow: /backup/ Disallow: /webalizer/ Disallow: /modlogan/ Disallow: /logs/ # Disallowed File Types ----------------------------------> Disallow: /*.css Disallow: /*.js Disallow: /wp-*.php Disallow: /*.zip Disallow: /*.exe Disallow: /*.pl Disallow: /*.cgi