Browse Source

block SemrushBot which is spamming our logs

pull/823/head
Max Mehl 2 years ago
parent
commit
dafb337ed1
Signed by: max.mehl GPG Key ID: 2704E4AB371E2E92
1 changed files with 3 additions and 0 deletions
  1. +3
    -0
      robots.txt

+ 3
- 0
robots.txt View File

@ -1,2 +1,5 @@
User-agent: *
Disallow: /source/
User-agent: SemrushBot
Disallow: /

Loading…
Cancel
Save