FSFE
/
fsfe-website
20
24
Fork 52
Code Issues 54 Pull Requests 8 Releases 0 Wiki Activity
Browse Source

block SemrushBot which is spamming our logs

pull/823/head
Max Mehl 2 years ago
parent
9fbd60bc1e
commit
dafb337ed1
Signed by: max.mehl GPG Key ID: 2704E4AB371E2E92
1 changed files with 3 additions and 0 deletions
Split View
Diff Options
Show Stats Download Patch File Download Diff File
  1. +3
    -0
      robots.txt

+ 3
- 0
robots.txt View File

@ -1,2 +1,5 @@
User-agent: *
Disallow: /source/
User-agent: SemrushBot
Disallow: /

Write Preview
Loading…
Cancel
Save