robots.txt:
No scraping, crawling, or systematic extraction of content
In short: Please use our site like a human, not a robot.
No use of BBC content for training or fine-tuning AI models, including large language models (LLMs)
No retrieval-augmented generation (RAG), AI-powered search, agentic AI or grounding using BBC content
No creating datasets from BBC content
No text and data mining (TDM) under Article 4 of the EU Directive on Copyright in the Digital Single Market
No using BBC content to create summaries for your own use
No business use without permission (details: https://www.bbc.co.uk/usingthebbc/terms/can-i-use-bbc-content-for-my-business/)
The BBC reserves all rights in its content and expressly opts out of any statutory exceptions in any jurisdiction for text and data mining, as permitted by law
| Agent | Rules it follows (RFC 9309) | Over the whole file |
|---|---|---|
| * | the * group | 49 Disallow rules; / allowed |
| GPTBot | its own group | disallowed at / (line 154), with 2 Allow rules; not bound by 1 of the * Disallow rules |
| OAI-SearchBot | its own group | disallowed at / (line 181) |
| ChatGPT-User | its own group | disallowed at / (line 159), with 2 Allow rules; not bound by 1 of the * Disallow rules |
| ClaudeBot | its own group | disallowed at / (line 133) |
| Google-Extended | its own group | disallowed at / (line 164), with 2 Allow rules; not bound by 1 of the * Disallow rules |
| Applebot-Extended | its own group | disallowed at / (line 151) |
| PerplexityBot | its own group | disallowed at / (line 169) |
| CCBot | its own group | disallowed at / (line 121) |
| Bytespider | its own group | disallowed at / (line 142) |
| meta-externalagent | its own group | disallowed at / (line 178) |
Not found by us: no API, feed or export.