How to generate robots.txt file in Kooomo
In robots txt panel, you are able to do the following:
• block all web crawlers from all content:
User-agent:* Disallow: /
• block a specific web crawler from a specific folder
User-agent: Google bot Disallow: /no-google/
• block a specific web crawler from a specific web page
User-agent: Google bot Disallow: /no-google/blocked-page.html
• Sitemap parameter User-agent:
* Disallow: Sitemap: http://www.example.com/none-standard-location/sitemap.xml