blog/_src/robots.txt

mirror of https://github.com/kremalicious/blog.git synced 2024-06-30 21:52:05 +02:00
User-agent: *
Disallow: /search/
Disallow: /page/
Disallow: /*/page/

Sitemap: {{ site.url }}/sitemap.xml
robots.txt, sitemap plugin 2013-11-21 00:44:52 +01:00			`User-agent: *`
			`Disallow: /search/`
			`Disallow: /page/`
also ignore category pagination 2013-12-11 22:55:39 +01:00			`Disallow: /*/page/`
robots.txt, sitemap plugin 2013-11-21 00:44:52 +01:00
better post date and updated date 2013-12-01 21:46:31 +01:00			`Sitemap: {{ site.url }}/sitemap.xml`