User-agent: * Disallow: /cgi-bin/ Disallow: /assets/ Disallow: /assets/images Disallow: /assets/css Disallow: /assets/script