# this tells all robots they can crawl all pages on the site User-agent: * Disallow: