meta-externalagent (Meta)
Model training · Current version 1.1
What it is
Meta-ExternalAgent crawls the web for use cases such as training foundation AI models or improving products by indexing content directly. Meta asks site owners to manage it through robots.txt. Meta AI search uses a different crawler, Meta-WebIndexer.
Official documentation: developers.facebook.com · checked 13 Nov 2026
meta-externalagent user agent
meta-externalagent/1.1 (+/documentation/sharing/webmasters/web-crawlers)meta-externalagent/1.1Official documentation: developers.facebook.com/docs/sharing/webmasters/web-crawlers · checked 13 Nov 2026
Block or allow meta-externalagent in robots.txt
Put these lines at the top of the robots.txt file in your site’s root (example.com/robots.txt). Each subdomain needs its own file.
User-agent: meta-externalagent
Disallow: /
User-agent: meta-externalagent
Allow: /
# repeat here the Disallow lines of your User-agent: * group
If meta-externalagent is not named in robots.txt, it already follows your User-agent: * rules, so you don’t need anything. A group of its own replaces the * group for this bot entirely: if you add one, repeat your general Disallow lines in it.
- Meta says its crawlers may cache robots.txt for up to 24 hours.
meta-externalagent IP ranges
Meta does not publish an IP list for meta-externalagent.
Does it respect robots.txt and Crawl-delay?
robots.txt: yes, meta-externalagent follows it according to its owner.
“In order to block these crawlers, add a disallow for the relevant crawler to robots.txt.” — Meta documentation
Crawl-delay: the owner’s documentation does not mention it for meta-externalagent, so don’t rely on it.
Should you block meta-externalagent?
It is a legitimate choice. Blocking meta-externalagent opts your content out of this kind of use, and search and AI answers that rely on other bots are not affected. If you want your content used by AI models, leave it open.
Check your site
See whether your robots.txt and firewall let meta-externalagent in — together with every other AI bot in this directory. Free, no sign-up.
Questions
What is the meta-externalagent user agent string?+
Meta documents it as: meta-externalagent/1.1 (+/documentation/sharing/webmasters/web-crawlers)
How do I block meta-externalagent in robots.txt?+
Add a group for it to robots.txt in your site’s root: “User-agent: meta-externalagent” followed by “Disallow: /”.
What IP addresses does meta-externalagent use?+
Meta does not publish an IP list for meta-externalagent.
Does meta-externalagent respect robots.txt?+
robots.txt: yes, meta-externalagent follows it according to its owner.