blog/static/robots.txt

---
---

User-agent: *
Disallow: /search/
Disallow: /page/
Disallow: /*/page/
Disallow: /tag/
Disallow: /api/

Sitemap: {{ site.url }}/sitemap.xml
fix sitemap and some 404s 2013-12-26 18:17:33 +01:00			`---`
			`---`

robots.txt, sitemap plugin 2013-11-21 00:44:52 +01:00			`User-agent: *`
			`Disallow: /search/`
			`Disallow: /page/`
also ignore category pagination 2013-12-11 22:55:39 +01:00			`Disallow: /*/page/`
dont index tags 2015-06-07 23:01:43 +02:00			`Disallow: /tag/`
kremalicious as a service - output all posts with all metadata as json 2016-05-02 01:05:53 +02:00			`Disallow: /api/`
robots.txt, sitemap plugin 2013-11-21 00:44:52 +01:00
dont index tags 2015-06-07 23:01:43 +02:00			`Sitemap: {{ site.url }}/sitemap.xml`