This website requires JavaScript.
Explore
Help
Sign in
pim
/
static
Archived
Watch
1
Star
0
Fork
You've already forked static
0
Code
Issues
Pull requests
Projects
Releases
Packages
Wiki
Activity
This repository has been archived on
2024-04-26
. You can view files and clone it, but cannot push or open issues or pull requests.
1cab65cfea
static
/
robots.txt
9 lines
136 B
Text
Raw
Normal View
History
Unescape
Escape
block semrush bot through robots.txt
2023-08-23 17:03:22 +00:00
User-agent: SemrushBot
Disallow: /
block AhrefsBot in robots.txt
2023-08-24 14:07:44 +00:00
User-agent: AhrefsBot
Disallow: /
update RP2 grading status block dotbot in robots.txt
2023-08-25 08:37:49 +00:00
User-agent: DotBot
Disallow: /
disallow facebookbot using robots.txt
2023-08-27 12:04:37 +00:00
User-Agent: FacebookBot
Disallow: /
Reference in a new issue
Copy permalink