Fetching the web with web_fetch

web_fetch is the MCP tool that turns a public URL into something a model can read: markdown for HTML pages, the file bytes for PDFs, and a passthrough for JSON, feeds, and plain text. It is available on every MCP session and needs only the mcp:read scope.

When to use it

Reach for web_fetch when the agent needs the content of a specific page it already knows the address of — a docs page, a changelog, an API reference, a public dataset, a PDF report. It is the right tool for "read this and answer" and "pull the facts from that page".

Do not use it as a search engine (it takes a URL, not a query), to call an API that needs credentials (use a connection or a skill), or to write anything anywhere: web_fetch is GET only, by design and permanently. Write verbs belong to connectors and skills, where the credential and the audit trail live.

What comes back

The tool returns a web_fetch_tool_result block in the same shape Anthropic's native web-fetch tool uses, so a Claude-family model reads it with no prompt work. The content is either a result or a typed error:

{
  "type": "web_fetch_tool_result",
  "content": {
    "type": "web_fetch_result",
    "url": "https://example.com/guide",
    "retrieved_at": "2026-09-12T10:00:00Z",
    "content": { "type": "document", "source": { "type": "text", "media_type": "text/plain", "data": "# Guide …" } },
    "next_start_index": 18234,
    "cache_hit": false,
    "content_bytes_original": 61230,
    "content_tokens_returned": 5000
  }
}

• HTML is extracted to markdown (navigation, scripts, and boilerplate stripped) with the page title.
• PDF is passed through as base64 for Claude-family models, which read documents natively. Other model families receive unsupported_content_type for PDFs today.
• JSON is returned inline when small, truncated with a note when medium, and summarised to its top-level keys when large.
• RSS / Atom / XML feeds are summarised to a title + item list.
• Plain text / CSV pass through under the token cap.

One redirect hop is followed. Private-network and loopback addresses are refused (url_not_allowed).

Pagination

Content is capped at max_tokens (default 5 000, up to 25 000). When the page is longer the result carries next_start_index; pass it back as start_index on the next call to continue from where the previous page ended. The same URL with a different start_index is a separate cache entry, so paging never returns page one twice.

Set use_cache: false only when freshness matters — a bypass always costs a real fetch.

Robots and origin policy

Before a page is fetched, the origin's robots.txt is consulted (cached per origin for 24 hours). A path the site disallows for our crawler token or for * is refused with blocked_by_robots, and the page itself is never requested. If robots.txt cannot be fetched the call proceeds — an outage on the origin's side is not a disallow.

Origins that rate-limit us surface as too_many_requests; other origin errors surface as url_not_accessible with the HTTP status in the hint.

Limits and metering

web_fetch is a paid tool: every call records a usage event against the account and is refused with insufficient_credits when the wallet is empty. On top of credits, each account has its own daily caps, read from the account's settings:

Setting	Default	Meaning
web_fetch.enabled	true	Hard off-switch; when false every call returns url_not_allowed.
web_fetch.daily_max_uses	500	Calls per UTC day.
web_fetch.daily_max_microusd	5 000 000 ($5)	Spend per UTC day.

When either cap is reached the tool returns max_uses_exceeded with a hint naming the cap. The slot is reserved before the fetch fires, so a rejected or failed call still counts — concurrent callers cannot race past the cap.

Error codes

Code	Meaning	What the agent should do
invalid_tool_input	Missing or malformed URL.	Fix the input.
url_too_long	URL over 2 048 characters.	Shorten or find a canonical URL.
url_not_allowed	Private network, or the tool is disabled for this account.	Do not retry.
blocked_by_robots	The origin's robots.txt disallows the path.	Do not retry; find another source.
url_not_accessible	Origin returned 4xx/5xx or was unreachable.	Try a different URL; retry later for 5xx.
too_many_requests	Origin returned 429.	Back off before retrying.
unsupported_content_type	Image, binary, or PDF for a non-Claude model.	Try content_purpose: metadata_only.
content_too_large	Body over the size ceiling.	Use pagination or a smaller resource.
max_uses_exceeded	A daily cap is reached.	Stop; the cap resets at UTC midnight.
insufficient_credits	The account wallet is empty.	Top up under Billing → Credits.
unavailable	The fetcher is not configured or failed internally.	Retry later.

Errors come back inside the result block, never as a protocol error, so the model can decide whether to retry.