This website requires JavaScript.
Explore
Help
Sign In
FSFE
/
fsfe-website
Watch
29
Star
30
Fork
You've already forked fsfe-website
85
Code
Issues
62
Pull Requests
14
Wiki
Activity
ff4f81e510
fsfe-website
/
robots.txt
6 lines
69 B
Plaintext
Raw
Normal View
History
Unescape
Escape
add robots.txt files for fsfe.org and test.fsfe.org
2018-12-10 13:25:20 +00:00
User-agent: *
Disallow: /source/
block SemrushBot which is spamming our logs
2019-03-06 12:46:52 +00:00
User-agent: SemrushBot
Disallow: /