ReplyfulBot

Documentation for ReplyfulBot, the web crawler that powers Replyful's AI knowledge base.

What is ReplyfulBot?

ReplyfulBot is the web crawler used by Replyful, an AI-powered customer support platform. It crawls websites to build a knowledge base that enables Replyful to provide accurate, AI-driven answers to customer questions.

ReplyfulBot only crawls domains that have been explicitly configured by site owners through the Replyful dashboard. It does not crawl websites without the site owner's authorization.

User-Agent

ReplyfulBot identifies itself with the following User-Agent:

Mozilla/5.0 (compatible; ReplyfulBot/1.0; +https://replyful.com/bot)

Crawl behavior

  • Rate limiting: Minimum 3-second delay between requests to the same domain
  • Concurrency: Maximum 3 concurrent requests per domain
  • robots.txt: Respects robots.txt directives
  • Scope: Only crawls domains explicitly configured by site owners via the Replyful dashboard

How to block ReplyfulBot

To prevent ReplyfulBot from crawling your site, add the following to your robots.txt file:

User-agent: ReplyfulBot
Disallow: /

You can also block specific paths while allowing others:

User-agent: ReplyfulBot
Disallow: /private/
Disallow: /admin/

Contact

If you have questions or concerns about ReplyfulBot, please contact us at [email protected].