How to generate robots.txt file in Kooomo


In robots txt panel, you are able to do the following:

• block all web crawlers from all content:
User-agent:* Disallow: /

• block a specific web crawler from a specific folder
User-agent: Google bot Disallow: /no-google/

• block a specific web crawler from a specific web page
User-agent: Google bot Disallow: /no-google/blocked-page.html

• Sitemap parameter User-agent:

* Disallow: Sitemap: http://www.example.com/none-standard-location/sitemap.xml