--- language: en title: "meta-externalfetcher" description: "meta-externalfetcher fetches links requested by users and supports Meta’s agentic capabilities. Learn about the possible exception to robots.txt rules." canonical: "https://tropk.ai/en/resources/glossary/meta-externalfetcher" last_modified: "2026-10-12T16:11:13.361Z" --- # meta-externalfetcher Reviewed on 2026-10-06. ## What it means meta-externalfetcher identifies Meta’s agent that fetches individual links at users’ request. The documentation also associates it with product functions, such as evaluating and improving agentic AI capabilities, including browsing websites to complete tasks. Because it performs user-requested fetches, the operator states that this agent may ignore [robots.txt](/en/resources/glossary/robots-txt) rules. ## Why it matters for marketing A marketing team may encounter this access when investigating why a page was fetched after an interaction with an assistant. Its purpose helps distinguish an individual request from broad [crawling](/en/resources/glossary/crawling). For restricted pages, relying only on robots.txt may leave a gap between the intent to control content and the documented behavior of meta-externalfetcher. ## Example in practice Imagine an agency testing a campaign preview link request in a Meta product. The preview does not require a login, even though its directory is disallowed in robots.txt. The server logs meta-externalfetcher receiving HTTP 200 for that URL. The agency identifies the lack of authentication on the preview and requests access control before sharing new versions. ## How to apply or verify In this proposed check, the infrastructure lead examines the same preview URL without an authenticated session, along with the HTTP response and server rule. Robots.txt communicates crawling preferences; protecting the preview depends on access control. The meta-externalfetcher header helps classify the log entry, but does not by itself prove that the request came from Meta. ## Important distinctions meta-externalfetcher has a different purpose from [meta-externalagent](/en/resources/glossary/meta-externalagent) and [meta-webindexer](/en/resources/glossary/meta-webindexer). Its possible exception to robots.txt does not authorize applying the same interpretation to the other agents. Fetching a link does not prove [citation](/en/resources/glossary/ai-citation), [indexing](/en/resources/glossary/indexing), or completion of the task. The reference to evaluating agentic capabilities also means its use cannot be summarized as a simple social preview; blocking it later does not show that previously retrieved content was removed. ## Sources and references - [Meta web crawlers]() — Meta ## Continue a leitura - [meta-externalagent]() - [meta-webindexer]() - [robots.txt]() ## About the author [Vinicius Pacheco]() Responsible for Tropk’s technical content, focusing on crawling, indexing, structured data, and agent access to content. --- [All pages in Markdown]()