pim/static
Archived
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity
This repository has been archived on 2024-04-26. You can view files and clone it, but cannot push or open issues or pull requests.
static/robots.txt

5 lines
69 B
Text
Raw Normal View History

block semrush bot through robots.txt
2023-08-23 17:03:22 +00:00
User-agent: SemrushBot
Disallow: /
block AhrefsBot in robots.txt
2023-08-24 14:07:44 +00:00
User-agent: AhrefsBot
Disallow: /
Reference in a new issue Copy permalink