Fetching the web with web_fetch

web_fetch is the MCP tool that turns a public URL into something a model can read: markdown for HTML pages, the file bytes for PDFs, and a passthrough for JSON, feeds, and plain text. It is available on every MCP session and needs only the mcp:read scope.

When to use it

Reach for web_fetch when the agent needs the content of a specific page it already knows the address of — a docs page, a changelog, an API reference, a public dataset, a PDF report. It is the right tool for "read this and answer" and "pull the facts from that page".

Do not use it as a search engine (it takes a URL, not a query), to call an API that needs credentials (use a connection or a skill), or to write anything anywhere: web_fetch is GET only, by design and permanently. Write verbs belong to connectors and skills, where the credential and the audit trail live.

What comes back

The tool returns a web_fetch_tool_result block in the same shape Anthropic's native web-fetch tool uses, so a Claude-family model reads it with no prompt work. The content is either a result or a typed error:

{
  "type": "web_fetch_tool_result",
  "content": {
    "type": "web_fetch_result",
    "url": "https://example.com/guide",
    "retrieved_at": "2026-09-12T10:00:00Z",
    "content": { "type": "document", "source": { "type": "text", "media_type": "text/plain", "data": "# Guide …" } },
    "next_start_index": 18234,
    "cache_hit": false,
    "content_bytes_original": 61230,
    "content_tokens_returned": 5000
  }
}

• HTML is extracted to markdown (navigation, scripts, and boilerplate stripped) with the page title.
• PDF is passed through as base64 for Claude-family models, which read documents natively. Other model families receive unsupported_content_type for PDFs today.
• JSON is returned inline when small, truncated with a note when medium, and summarised to its top-level keys when large. With content_purpose: download it is the raw JSON, never summarised, paginated with start_index like any text — read every page and the pieces join into the original document.
• RSS / Atom / XML feeds are summarised to a title + item list.
• Plain text / CSV pass through under the token cap.

Redirects (301, 302, 303, 307, 308) are followed, up to 5 of them. Each hop is checked like the first URL: it must be https://, its origin's robots.txt must allow the path, and a private-network, loopback or metadata address is refused. A redirect to http:// or to a private address ends the call with url_not_allowed; a hop that robots.txt disallows ends it with blocked_by_robots; a sixth redirect ends it with url_not_accessible. The result's url is the page that was finally fetched, not the URL you passed.

Pagination

Content is capped at max_tokens (default 5 000, up to 25 000). When the page is longer the result carries next_start_index; pass it back as start_index on the next call to continue from where the previous page ended. Both count characters of the extracted text, not bytes. A hosted agent's call is capped lower (about 7 000 tokens) so a page fits in one tool result and is paged, not cut; a PDF too large for one tool result comes back as content_too_large. The same URL with a different start_index is a separate cache entry, so paging never returns page one twice.

Set use_cache: false only when freshness matters — a bypass always costs a real fetch.

Robots and origin policy

Before a page is fetched, the origin's robots.txt is consulted (cached per origin for 24 hours). A path the site disallows for our crawler token or for * is refused with blocked_by_robots, and the page itself is never requested. If robots.txt cannot be fetched the call proceeds — an outage on the origin's side is not a disallow.

Origins that rate-limit us surface as too_many_requests; other origin errors surface as url_not_accessible with the HTTP status in the hint.

Limits and metering

web_fetch is a paid tool: every fetch records a usage event against the account and is refused with insufficient_credits when the wallet is empty. A call that ends in an error result (blocked_by_robots, url_not_allowed, …) fetched nothing and records none. On top of credits, each account has its own daily caps, read from the account's settings. They apply to the MCP tool and to a hosted agent's web-fetch skill alike:

Setting	Default	Meaning
web_fetch.enabled	true	Hard off-switch; when false every call returns url_not_allowed.
web_fetch.daily_max_uses	500	Calls per UTC day.
web_fetch.daily_max_microusd	5 000 000 ($5)	Spend per UTC day; each fetch adds its price from the pricebook.

When either cap is reached the tool returns max_uses_exceeded with a hint naming the cap. The slot is reserved before the fetch fires, so a rejected or failed call still counts — concurrent callers cannot race past the cap.

Error codes

Code	Meaning	What the agent should do
invalid_tool_input	Missing or malformed URL.	Fix the input.
url_too_long	URL over 2 048 characters.	Shorten or find a canonical URL.
url_not_allowed	Private network, not https:// (the URL or a redirect), or the tool is disabled for this account.	Do not retry.
blocked_by_robots	The origin's robots.txt disallows the path.	Do not retry; find another source.
url_not_accessible	Origin returned 4xx/5xx, was unreachable, or redirected more than 5 times.	Try a different URL; retry later for 5xx.
too_many_requests	Origin returned 429.	Back off before retrying.
unsupported_content_type	Image, binary, or PDF for a non-Claude model.	Try content_purpose: metadata_only.
content_too_large	Body over the size ceiling.	Use pagination or a smaller resource.
max_uses_exceeded	A daily cap is reached.	Stop; the cap resets at UTC midnight.
insufficient_credits	The account wallet is empty.	Top up under Billing → Credits.
unavailable	The fetcher is not configured or failed internally.	Retry later.

Errors come back inside the result block, never as a protocol error, so the model can decide whether to retry.