crawlsignals · How we read · About, corrections and removal · Privacy · records.json

Wikipedia en.wikipedia.org · reference

What it says about automated access

robots.txt:

robots.txt https://en.wikipedia.org/robots.txt read 2026-10-03

AgentRules it follows (RFC 9309)Over the whole file
*the * group428 Disallow rules; / allowed

With no group of their own, these follow the * group: Googlebot, Bingbot, GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, Claude-SearchBot, Claude-User, Google-Extended, Applebot-Extended, PerplexityBot, CCBot, Bytespider, meta-externalagent.

Official way in

Documents

Wikipedia:Text of the Creative Commons Attribution-ShareAlike 4.0 International License licence · linked from https://en.wikipedia.org/ as "Creative Commons Attribution-ShareAlike 4.0 License" · read 2026-10-03

Terms of Use terms · linked from https://en.wikipedia.org/ as "Terms of Use" · not read by us: robots.txt of foundation.wikimedia.org disallows it for our crawler

Wikimedia Foundation Legal and Safety Contact Information other · linked from https://en.wikipedia.org/ as "Legal & safety contacts" · not read by us: robots.txt of foundation.wikimedia.org disallows it for our crawler

Wikimedia Developer Portal api-docs · linked from https://en.wikipedia.org/ as "Developers" · read 2026-10-03

Get started api-docs · read 2026-10-03

Terms of Use terms · linked from https://developer.wikimedia.org/ as "Terms of Use" · not read by us: robots.txt of foundation.wikimedia.org disallows it for our crawler

GNU General Public License licence · linked from https://developer.wikimedia.org/ as "Code: GPL-3.0-or-later" · read 2026-10-03

Attribution-ShareAlike 4.0 International licence · linked from https://developer.wikimedia.org/ as "Content: CC-BY-SA-4.0" · read 2026-10-03

Developer Portal other · linked from https://developer.wikimedia.org/ as "About" · not read by us: robots.txt of www.mediawiki.org disallows it for our crawler

Developer Portal/Contribute other · linked from https://developer.wikimedia.org/ as "Contribute" · not read by us: robots.txt of www.mediawiki.org disallows it for our crawler

Nothing here is legal advice. A mistake? Write to hello@crawlsignals.com (corrections).