Leilukin/leilukin-tumbleblog
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Add imgproxy, NovaAct and Operator

Browse Source
This commit is contained in:
Helen Chong 2025-04-19 07:28:55 +08:00
parent 51f9e3f710
commit 111ea627df
2 changed files with 6 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
.htaccess
View File

@ -20,5 +20,5 @@ RewriteRule ^.+$ index.php [L]
</IfModule> </IfModule>
# Block bad bots # Block bad bots
RewriteCond %{HTTP_USER_AGENT} (AI2Bot|Ai2Bot-Dolma|Amazonbot|anthropic-ai|Applebot-Extended|Brightbot.*1.0|Bytespider|ChatGPT-User|Claude-Web|ClaudeBot|cohere-ai|cohere-training-data-crawler|Diffbot|DuckAssistBot|FacebookBot|FriendlyCrawler|Google-Extended|GoogleOther|GoogleOther-Image|GoogleOther-Video|GPTBot|iaskspider/2.0|ICC-Crawler|ImagesiftBot|img2dataset|ISSCyberRiskCrawler|Kangaroo.*Bot|Meta-ExternalAgent|Meta-ExternalFetcher|OAI-SearchBot|omgili|omgilibot|PanguBot|PerplexityBot|PetalBot|Scrapy|SemrushBot-OCOB|SemrushBot-SWA|Sidetrade.*indexer.*bot|Timpibot|VelenPublicWebCrawler|Webzio-Extended|YouBot) [NC] RewriteCond %{HTTP_USER_AGENT} (AI2Bot|Ai2Bot-Dolma|Amazonbot|anthropic-ai|Applebot-Extended|Brightbot.*1.0|Bytespider|ChatGPT-User|Claude-Web|ClaudeBot|cohere-ai|cohere-training-data-crawler|Diffbot|DuckAssistBot|FacebookBot|FriendlyCrawler|Google-Extended|GoogleOther|GoogleOther-Image|GoogleOther-Video|GPTBot|iaskspider/2.0|ICC-Crawler|ImagesiftBot|img2dataset|imgproxy|ISSCyberRiskCrawler|Kangaroo.*Bot|Meta-ExternalAgent|Meta-ExternalFetcher|NovaAct|OAI-SearchBot|omgili|omgilibot|Operator|PanguBot|PerplexityBot|PetalBot|Scrapy|SemrushBot-OCOB|SemrushBot-SWA|Sidetrade.*indexer.*bot|Timpibot|VelenPublicWebCrawler|Webzio-Extended|YouBot) [NC]
RewriteRule .* https://nocommercialuse.org/ [L] RewriteRule .* https://nocommercialuse.org/ [L]

3
robots.txt
View File

@ -26,13 +26,16 @@ User-agent: iaskspider/2.0
User-agent: ICC-Crawler User-agent: ICC-Crawler
User-agent: ImagesiftBot User-agent: ImagesiftBot
User-agent: img2dataset User-agent: img2dataset
User-agent: imgproxy
User-agent: ISSCyberRiskCrawler User-agent: ISSCyberRiskCrawler
User-agent: Kangaroo Bot User-agent: Kangaroo Bot
User-agent: Meta-ExternalAgent User-agent: Meta-ExternalAgent
User-agent: Meta-ExternalFetcher User-agent: Meta-ExternalFetcher
User-agent: NovaAct
User-agent: OAI-SearchBot User-agent: OAI-SearchBot
User-agent: omgili User-agent: omgili
User-agent: omgilibot User-agent: omgilibot
User-agent: Operator
User-agent: PanguBot User-agent: PanguBot
User-agent: PerplexityBot User-agent: PerplexityBot
User-agent: PetalBot User-agent: PetalBot