robots.txt:
If you're a bot who likes crawling webpages, please mind the house rules and avoid accessing any disallowed subfolders to earn a 5-star review from us.
| Agent | Rules it follows (RFC 9309) | Over the whole file |
|---|---|---|
| * | the * group | 61 Disallow rules; / allowed |
| Googlebot | its own group | 61 Disallow rules; / allowed |
| Bingbot | its own group | 61 Disallow rules; / allowed |
| GPTBot | its own group | disallowed at / (line 691) |
| OAI-SearchBot | its own group | 61 Disallow rules; / allowed |
| ChatGPT-User | its own group | 61 Disallow rules; / allowed |
| ClaudeBot | its own group | disallowed at / (line 695) |
| Applebot-Extended | its own group | disallowed at / (line 699) |
| PerplexityBot | its own group | 63 Disallow rules; / allowed |
| meta-externalagent | its own group | 61 Disallow rules; / allowed |
Not found by us: no API, feed or export.
Not found by us: terms of use.