Remove default crawler block

Move up sitemap in robots.txt
2024-08-20 08:38:24 +08:00 · 2024-08-20 08:35:09 +08:00
1 changed files with 4 additions and 4 deletions
--- a/src/robots.txt.njk
+++ b/src/robots.txt.njk
@ -2,8 +2,10 @@
 permalink: /robots.txt
 eleventyExcludeFromCollections: true
 ---
+Sitemap: {{ sitemeta.siteUrl }}/sitemap/
+
 User-agent: *
-Disallow: /
+Disallow:

 User-agent: Amazonbot
 User-agent: Applebot
@ -39,5 +41,3 @@ User-agent: img2dataset
 User-agent: omgili
 User-agent: omgilibot
 Disallow: /
-
-Sitemap: {{ sitemeta.siteUrl }}/sitemap/
Author	SHA1	Message	Date
Helen Chong	6300906b81	Remove default crawler block	2024-08-20 08:38:24 +08:00
Helen Chong	85337c4d33	Move up sitemap in robots.txt	2024-08-20 08:35:09 +08:00