# Hash preceeds a comment # To exclude certain pages specifically add this meta tag into the head # # All robots will spider the domain User-agent: * Disallow: