FSFE/fsfe-website
25
31
Fork 91
Code Issues 64 Pull Requests 5 Wiki Activity
fsfe-website/robots.txt

6 lines
69 B
Plaintext
Raw Normal View History

add robots.txt files for fsfe.org and test.fsfe.org
2018-12-10 14:25:20 +01:00
User-agent: *
Disallow: /source/
block SemrushBot which is spamming our logs
2019-03-06 13:46:52 +01:00
User-agent: SemrushBot
Disallow: /
Reference in New Issue Copy Permalink