About CoachCompareBot
If you found this page in your server logs: this is who visited, what it read and how to stop it.
Who runs it
Coach Compare is a comparison site for bodybuilding coaches. To show what coaches charge, it reads the prices that coaches publish on their own public pages.
How to recognise it
Every request carries this user agent:
CoachCompareBot/0.1 (+https://coachcompare.fyi/bot)
What it does
- Reads public coach websites and public package pages on coaching platforms.
- Looks for prices, what a package includes, and a coach's stated qualifications.
- Makes at most one request every 10 seconds to a site, or slower if your robots.txt sets a Crawl-delay. It reads at most 8 pages on a visit, and normally visits no more than once every 30 days.
- Keeps a copy of each page it reads for up to 90 days, so that a price can be checked against the page it came from. Then the copy is deleted.
What it respects
- robots.txt. A page you disallow is never requested.
- Rules for AI crawlers. We use an AI model to read prices, so if your robots.txt turns away GPTBot, ClaudeBot or the like by name, we take that to include us.
noindex,noarchive,nosnippetandnoaitags, and atdm-reservationheader or tag, which reserves your text against data mining. A page that carries one is not kept.nofollowis honoured too.- A refusal. If your site answers 401, 402, 403 or 429, or shows a "prove you are human" page, the visit stops there and the crawler does not come back by itself.
What it never does
- It never visits Instagram, Facebook or Threads.
- It never logs in, fills in forms, sends cookies or goes behind a paywall.
- It never pretends to be a browser or another crawler, and never retries under a different name.
- It downloads no images or video, keeps away from pages about other people (testimonials, client results, galleries), and never takes clients' names from a page.
How to stop it
Add this to your robots.txt and it will stay away:
User-agent: CoachCompareBot Disallow: /
To have what we already hold removed as well, use the correction and removal form. A removal also blocks your site from being read again.
Contact
Questions about the crawler: hello@coachcompare.fyi