Description:
Blocks selected bots from indexing threads.
(Example of Options page)
Features:
All phrases start with blockbots_ for your convenience.
Questions and answers:
Q: What error code is give to the bot?
A: An error code 429 is give (Too Many Requests) and a Retry-After of 24 hours.
Q: Should I remove the User agent token after 24 hours?
A: Yes. Good bots will do an internal adjustment and when they return will start crawling your forum with fewer request in order to not slow down your server.
Blocks selected bots from indexing threads.
(Example of Options page)
Features:
All phrases start with blockbots_ for your convenience.
Questions and answers:
Q: What error code is give to the bot?
A: An error code 429 is give (Too Many Requests) and a Retry-After of 24 hours.
Q: Should I remove the User agent token after 24 hours?
A: Yes. Good bots will do an internal adjustment and when they return will start crawling your forum with fewer request in order to not slow down your server.
