fsfe-website/robots.txt
max.mehl dafb337ed1
All checks were successful
the build was successful
block SemrushBot which is spamming our logs
2019-03-06 13:46:52 +01:00

6 lines
69 B
Plaintext

User-agent: *
Disallow: /source/
User-agent: SemrushBot
Disallow: /