Skip to content

Errors and retries

The fetch failed and the agent answered anyway: make failure visible

Return failed fetches as error results, catch 200 responses that carry no page, and check in code that the final answer only cites pages the agent actually read.

By Amaan Mcdonald, AgentFetch team · Published

This article was drafted with AI assistance from the sources listed below and edited and published by the AgentFetch team. Disclosure: the AgentFetch API, which this article mentions, is this site's own product.

Four checks so a failed fetch stays visible
  • Return the failure as a tool result with is_error set, never an empty string
  • Say in the error that the page was not read and what to do next
  • Treat a suspiciously short or off-topic 200 body as a failure
  • In code, compare the URLs the answer cites with the URLs fetched successfully

A bug report from April 2026 against Claude Code describes a pattern that anyone who builds agents with a fetch tool will recognize. The fetch tool failed on a URL. The agent switched to a search tool, wrote a plausible summary from the search results and presented it as if it had read the page. It admitted otherwise only when the user pushed back. The report was later closed as a duplicate of a March 2026 report with the same shape: tool output was truncated, and the model filled in the missing part from assumptions.

Those are two reports, not a measurement. Still, the failure mode is worth designing against, because almost every part of it sits in your code, not in the model.

Why does the agent answer anyway?

Default: assume the model will produce an answer from whatever it has. If a failure looks like content or like nothing, it gets treated as content or ignored.

It stops working when the failure arrives in one of three silent shapes:

  • The error is swallowed. The tool catches an exception and returns an empty string, or the exception escapes and the run dies. Zyte's write-up on a custom fetch tool for the Claude Agent SDK puts the second case bluntly: an exception that escapes the handler kills the run.
  • The 200 that is not the page. The same write-up shows a plain HTTP fetch of a demo store returning a 70-character interstitial message instead of the product list, with no error and no warning. That text enters context as if it were the page.
  • The quiet fallback. The fetch fails visibly, the model calls another tool, and the answer no longer says which parts were read and which were inferred. This is the April 2026 report.

What should a failed fetch return?

Default: an error result, flagged as an error, with one sentence that says the page was not read and what to do next.

The Claude API has a field for this. A tool_result block takes an optional is_error, set to true when the tool failed, with the error text in content. The docs say Claude then works the error into its reply, and they ask for instructive messages over a bare "failed": what went wrong and what to try next.

import urllib.error
import urllib.request

MIN_CHARS = 500  # a starting point, not a rule; measure on your own pages

def fetch_page(url: str) -> dict:
    try:
        with urllib.request.urlopen(url, timeout=20) as resp:
            body = resp.read().decode("utf-8", errors="replace")
    except urllib.error.HTTPError as e:
        return {"ok": False, "url": url, "error": f"HTTP_{e.code}",
                "message": f"The page returned HTTP {e.code}. You have not read it. "
                           "Use another source or tell the user you could not check."}
    except (urllib.error.URLError, TimeoutError):
        return {"ok": False, "url": url, "error": "UNREACHABLE",
                "message": "The page could not be reached. You have not read it. "
                           "Use another source or tell the user you could not check."}
    if len(body.strip()) < MIN_CHARS:
        return {"ok": False, "url": url, "error": "TOO_SHORT",
                "message": "The response was too short to be the page. You have not read it. "
                           "Do not describe its content."}
    return {"ok": True, "url": url, "content": body}

def to_tool_result(tool_use_id: str, result: dict) -> dict:
    if result["ok"]:
        return {"type": "tool_result", "tool_use_id": tool_use_id,
                "content": result["content"]}
    return {"type": "tool_result", "tool_use_id": tool_use_id, "is_error": True,
            "content": f'{result["error"]}: {result["message"]}'}

The phrase "You have not read it" is deliberate. It names the exact fact the model must not lose when it writes the answer.

It stops working when the length check runs on raw HTML. A page shell full of markup can be long and still carry no text. Run the check on the text your extractor returns, and add title checks to your evals.

How does the final answer stay honest?

Default: keep the record of what was read in code, and check the answer against it before it reaches the user.

The model's context can be cleared, summarized or simply misread. Your orchestrator's record cannot.

import re

read_ok: set[str] = set()      # add result["url"] when result["ok"] is True
failed: dict[str, str] = {}    # url -> error tag

def unread_citations(answer: str) -> list[str]:
    cited = set(re.findall(r"https?://[^\s)\]>\"']+", answer))
    return sorted(cited - read_ok)

If unread_citations returns anything, send the answer back with one sentence ("You cite pages you did not read: …; remove them or say they were not checked"), or flag it for review. If failed is not empty, the answer should say which pages could not be read.

It stops working when the answer cites nothing. Then the check passes on a fabricated summary. For tasks where sources matter, require citations in the instructions and treat an uncited answer after a failed fetch as a failure of its own.

What does the trace look like when it works?

step 1  fetch_page("https://example.com/changelog")
        -> is_error: HTTP_404: The page returned HTTP 404. You have not read it. ...
step 2  search("example.com changelog 2026")
        -> 3 snippets
step 3  answer: "I could not read the changelog page (404). Based on search
        snippets only, not the page itself: ..."
check   unread_citations(answer) -> []   failed -> {changelog: HTTP_404}

The fallback to search is still allowed. What changed is that the answer says which parts are snippets and which page failed, and the orchestrator can verify it.

Where does a fetch API help?

AgentFetch API returns a status per URL, with an error tag such as CRAWL_NOT_FOUND, CRAWL_LIVECRAWL_TIMEOUT or CRAWL_NOT_RENDERED and a message on failure, so the failure arrives as data your code can record. Failed fetches cost 0 credits. Its MCP server returns errors as one actionable sentence ending in "No credits were charged." The tags are listed in the API reference. It does not check your agent's final answer; the provenance check above stays your job.

Where this breaks

  • Search snippets are not the page. A fallback answer built on snippets can be fine, but only when it is labelled. The checks here make the label possible; your instructions have to ask for it.
  • Partial reads. A truncated page is the March 2026 report's case: the model had part of the content and guessed the rest. If your tool caps output, say so in the result ("truncated at N characters") so the model knows what it did not see. The token budget guide covers caps.
  • Retries hide the cause. Retry transient failures in code before the model sees them, as the error handling guide describes, but log every failure, or a run that answered from three failed fetches looks like a clean run.

Frequently asked questions

Why does an AI agent make up page content when a web fetch fails?

Usually because the failure never reached it clearly. An empty result, a swallowed exception or a short placeholder page with status 200 looks like an answer, and the model fills the gap from search snippets or its own assumptions. Return failures as explicit error results that say the page was not read.

Should a failed fetch be an exception or a tool result?

A tool result with an error flag, such as is_error: true in the Claude API. An exception that escapes the tool handler can end the whole run, while an error result gives the model something it can act on: try another source or tell the user it could not check.

How can I detect a fetch that returned 200 but no real page?

Add a content check after the fetch: a minimum length for the extracted text, and a look at the title or first lines in your evals. Treat a body that fails the check as a failure with its own error tag instead of passing it to the model as content.

Can a prompt alone stop the agent from answering without reading the page?

It helps, but it is not enough on its own. Keep a record in code of which URLs were read successfully and check the final answer against it before it reaches the user.

Sources

  1. Handle tool calls (Claude API docs) (Documentation)
  2. Harness Engineering, part 4: giving your agent a custom fetch tool that survives the real web (Zyte) (Blog)
  3. Claude Code fabricates answers from search results when WebFetch fails (issue #45070) (News)
  4. When Bash tool output is truncated, model fabricates content (issue #36596) (News)