# /robots.txt file from http://www.robotstxt.org/ #User-agent: webcrawler #Disallow: /tmp #Disallow: /logs # #User-agent: lycra #Disallow: / # Allow everything from all bots. (At least one Disallow is required, so allow all means a blank entry..) User-agent: * Disallow: