#robots.txt file for http://www.thepiepeddler.com/ #allows know trusted bots to spider everything except for controlled directories #all else is blocked from traversing site from root User-agent: * Disallow: