GET/web/extract

Extract API

Fetch a public URL and return its content as text, markdown or HTML. `format=text` returns `text/plain; charset=utf-8`, `format=markdown` returns `text/markdown; charset=utf-8` and `format=html` returns `text/html; charset=utf-8`. `format=markdown` keeps headings, lists, tables and code: the main content of articles and docs, and the whole page without its navigation otherwise. By default `format=text` returns the page's main content (the article or main block, the way reader views do), which can leave out other sections such as tables, notices and sidebars; set `full_page=true` to get all visible text of the page instead, one block per line, with table cells separated by ` | `. Pages that cannot be served answer with an `error_code` and are not charged. Billing metadata: successful billable responses include the `X-Desearch-Cost-Usd`, `X-Desearch-Usage-Count`, `X-Desearch-Service`, and `X-Desearch-Currency` headers. JSON object responses also include optional `cost_usd`, `usage_count`, `service`, and `currency` fields. Current prices are at https://desearch.ai/pricing. JSON arrays, text, and streaming responses keep their body shape and expose billing metadata only through headers.

Base URL

https://api.desearch.ai

Implementation guidance

Use Extract API when your application needs fresh web resultswithout maintaining its own crawler, social-data pipeline, or retrieval infrastructure. The endpoint returns structured data that can be consumed by agents, research tools, dashboards, and retrieval-augmented generation workflows. Keep the request focused on the minimum fields your workflow needs so responses remain fast and easy to validate.

Send requests to GET /web/extract with your Desearch API key in the authorization header. Treat the key as a server-side secret, validate all required parameters before sending the request, and handle non-2xx responses explicitly. Production integrations should use sensible timeouts, retry only transient failures with backoff, and record the response status and request identifier for troubleshooting.

Test the example request below before integrating it into a larger workflow. Confirm that the returned fields, links, and timestamps meet your freshness requirements, then add schema validation at the application boundary. For agentic systems, preserve source URLs with the generated answer so users can inspect evidence and your application can avoid presenting unsupported claims.

Query Parameters

urlstringrequired

URL to extract content from

formatstring

Format of the content to be returned: 'text', 'markdown' or 'html'. Default is 'text'. 'text' is sent as text/plain, 'markdown' as text/markdown and 'html' as text/html.

Default: text

jsstring

Render the page with a headless browser before extracting content. Use for SPAs or pages whose body is populated by JavaScript. Must be exactly `true` or `false` (case-sensitive); enables a separate billing tier when set to `true`.

Default: false

waitstring

Optional extra wait in milliseconds after the page loads. Only applied when ``js=true``.

full_pagestring

With `format=text`, return all visible text of the page instead of its main content. Must be exactly `true` or `false`. Applies only to `format=text`; `markdown` picks the main content or the whole page itself.

Default: false

Responses

200Extracted content of the web link

301Moved Permanently

304Not Modified

401Unauthorized

404Not charged. `error_code` is `page_not_found` when the site answered 404 or 410, or `no_content` when the page has no text.

422Not charged. Either a validation error, or `error_code` `js_required`: the page only shows its content after JavaScript runs (an app shell, or a page that asks to enable JavaScript). Retry with `js=true`.

429Too Many Requests

500Internal Server Error

503Not charged. `error_code` is `blocked` when the site refused the request or answered with a bot check (a CAPTCHA or "verifying your request" page) instead of the page, `upstream_timeout` when the page took too long, and `upstream_failed` for any other crawler failure.

Authorization

Your API key is sent in the Authorization header.

Request

curl -X GET "https://api.desearch.ai/web/extract?url=https%3A%2F%2Fen.wikipedia.org%2Fwiki%2FArtificial_intelligence" \
  -H "Authorization: YOUR_API_KEY" \
  -H "Content-Type: application/json"

ResponseSchema

{
  "message": "Response schema not available"
}

Possible status codes:

200401422429500