API reference

Endpoint

POST https://livewebdata.com/api/v1/contents
Authorization: Bearer <api_key>
Content-Type: application/json

Request parameters

{
  "urls": ["https://example.com/article"],
  "countryCode": "de",
  "text": { "maxCharacters": 50000, "verbosity": "compact" },
  "livecrawlTimeout": 45000
}

Unknown fields are rejected with 400 INVALID_REQUEST; the message names the field. So is a URL longer than 8,192 characters (urls.0: Each URL may be at most 8,192 characters.).

Response

Browser fallback is on by default: a page the fast fetch cannot read (no or very little visible text, blocked, 401/403, a timeout) is rendered in a real browser and charged only the render credits. statuses[].fallbackUsed is true when that happened (strategy: "browser" = the content is the render). Send renderFallback: false for a fast fetch only. A page that still shows fewer than 500 characters of visible text after rendering is delivered as a success with statuses[].thin: true at 0 credits, holding only that visible text — it is not an error.

text.maxCharacters (default 100,000, max 1,000,000) cuts text and markdown; results[].textTruncated is true when it did. A response carries at most 24 MiB of JSON: from the result that would pass it on, results come without html, and if that is not enough, text and markdown are cut (textTruncated: true), each such result flagged RESPONSE_SIZE_LIMIT. The request never fails because of it. title and author are at most 2,000 characters each; an image or favicon URL longer than 4,096 characters is left out (favicon then falls back to /favicon.ico). The MCP tool get_page_contents has its own, smaller default of 8,000 (max_characters), because its result goes into an agent's context. results[].warnings is present only when there is something to know about a page:

Response headers

Errors

Per-URL errors

Returned inside statuses[].error with HTTP 200. These never cost credits.

Request errors

Returned with a non-200 status and no results.

{
  "error": {
    "code": 402,
    "tag": "INSUFFICIENT_CREDITS",
    "message": "Not enough credits. Top up or upgrade your plan."
  },
  "requestId": "req_01J…"
}

Long requests and Idempotency-Key

Most requests answer within seconds. A request still fetching after about 100 seconds (slow sites, rendering, a long livecrawlTimeout) answers 202 instead of making you wait on an open connection:

{
  "status": "pending",
  "requestId": "req_01J…",
  "idempotencyKey": "lwda_3f1c2b7e-0d4a-4c1e-9a55-2b8f0e6c7d10",
  "retryAfter": 1,
  "message": "Still fetching. Send the same request again with the same Idempotency-Key header to get the result; it is not charged again."
}

Send the same request body again with the header Idempotency-Key: <idempotencyKey> after Retry-After seconds. The repeat waits for the fetch and answers 200 with the result, or 202 again if it is still running. A repeat is never charged again. The SDKs and the MCP server do this for you.

Always send your own Idempotency-Key (8–128 characters of A–Z a–z 0–9 _ - : ., for example a UUID) on the first request, and keep it until you have the answer. A connection can break after the API received the request: a network error, a timeout on your side, a deploy. Send the same request again with the same key, and you get that fetch (its result, or 202 while it still runs) instead of a new one, so you pay once. Without your own key, a cut connection is a lost answer: the retry has no key, so it is a new request and charged again.

Every answer of POST /v1/contents (200, 202 and errors) carries the request's key in the Idempotency-Key response header: yours, or one the API created when you sent none. The only exceptions are the 400 for a malformed key and the 413 for a request body over 10 MiB, which is refused before it is read (nothing was charged, there is nothing to repeat). The SDKs create a key for each contents() call when you pass none, send it on every retry and put it on the error they raise (idempotencyKey / idempotency_key).

• Results are kept for 15 minutes. A result nobody collects stays charged; after 15 minutes a repeat answers 409 IDEMPOTENCY_CONFLICT, and a new key fetches the pages again.
• A key is bound to its request body for 24 hours: reusing it with a different body answers 409 IDEMPOTENCY_CONFLICT. Use a new key for every new request.

Rate limits

Limits are per account and count URLs per second (a request with 3 URLs uses 3). When exceeded, the API returns 429 with a Retry-After header (seconds). Limits work as a bucket: X-RateLimit-Limit is how many URLs you can send at once (rateLimit.burst in GET /v1/account), X-RateLimit-Remaining how many right now, and the bucket refills at your plan's URLs per second (rateLimit.requestsPerSecond).

Credits

• Plan credits reset monthly.
• Top-up credits never expire.

When your balance is too low for the URLs in a request, the API returns 402 INSUFFICIENT_CREDITS. The current balance is returned in X-Credits-Remaining on every response.

Account

GET https://livewebdata.com/api/v1/account returns your plan, remaining credits and rate limit. It is free — call it before large jobs.

{
  "plan": "starter",
  "status": "active",
  "credits": { "plan": 81234, "topup": 10000, "total": 91234 },
  "rateLimit": { "requestsPerSecond": 2, "burst": 4 },
  "periodEndsAt": "2026-11-01T00:00:00Z"
}