Search evaluations · run by Desearch

    Search quality, run by run

    Four public question sets and their published measures. See what each provider returned, where Desearch leads, and where another provider does better. The code and per-question evidence are public.

    Published snapshot
    19 Sept 2026
    Runs
    4
    Dataset revision
    d9d96b6eca9d
    Owner
    Desearch

    Published scoreboards

    Results and definitions

    Choose a dataset to compare published provider results. Retrieval measures and agent answer accuracy stay separate, with their own question counts.

    Explore the evidence

    01 / 04

    Search retrieval

    SimpleQA Verified

    463 questions
    Chart measure
    Reference page found in top 10
    Higher is better · 0–100%

    Each bar is a published provider mode. Desearch ran this evaluation.

    All published measures and modes

    Scroll the table sideways to see every measure.

    SimpleQA Verified: reference page found in top 10 and other published measures for 463 questions
    Provider · modeReference @1Reference @5Reference @10Page states answer @10Returned text states answer @10
    Desearch · fast90.9%98.3%99.1%96.1%86.6%
    Desearch · balanced93.1%98.7%99.1%96.1%86.6%
    Parallel · basic57.0%85.1%90.7%97.2%95.9%
    Parallel · fast32.4%79.5%89.2%98.1%97.4%
    Perplexity · search67.6%84.9%89.0%99.1%98.9%
    Exa · auto47.5%73.0%79.5%95.7%95.0%
    Exa · fast46.4%73.0%78.6%95.9%95.7%
    Tavily · basic61.1%63.3%63.5%71.9%71.3%
    Tavily · fast32.4%36.7%37.8%83.2%82.5%
    Open this run’s raw scoreboard ↗

    Method and evidence

    Read the result behind each number

    The three retrieval runs ask each provider profile the same questions and compare URLs with the benchmark reference pages. The SimpleQA Verified and SealQA exports additionally score whether fetched pages and returned text state the verified answer. FRAMES did not run that answer judgment, so its zero-valued placeholder fields are not answer scores. BrowseComp gives an agent the search tool and grades its final answer on a 100-question sample.

    19 Sept 2026 is the dataset snapshot publication date, not a claimed date for every search run. Desearch ran the evaluations; the result files and code can be audited independently.

    Product comparison

    API capabilities

    A separate comparison of documented product capabilities, reviewed 21 September 2026. Select a cell for the provider's source. These entries are not benchmark results.

    Swipe sideways to compare all providers →

    Desearch
    Web search
    Dedicated social APIs
    AI answer + source links
    Extract a supplied URL
    Crawl linked site pages
    MCP server
    Cost or usage in response
    Reviewed 21 Sep 2026 from public provider docs. Social means dedicated X, TikTok, Facebook and Instagram APIs; crawl means following linked pages. Answer and source links may be separate fields. Not established does not mean absent. USD costs and vendor credits are different units.
    Documented API capabilities as reviewed 21 September 2026; not established does not mean absent
    CapabilityDesearchTavilyExaSerpApiPerplexityFirecrawl
    Web searchShips. Web search results via /web.Ships. Search API returns ranked web results.Ships. Search API supports neural and keyword retrieval.Ships. Search API returns results from supported engines.Ships. Search API returns ranked web results.Ships. Search endpoint returns web results.
    Dedicated social APIsShips. Dedicated X, TikTok, Facebook and Instagram endpoints.Not established in reviewed docs. No dedicated platform endpoints identified in the reviewed Search API docs; web results may still include social pages.Not established in reviewed docs. No dedicated platform endpoints identified in the reviewed Search API docs; indexed social pages are a different capability.Partial. Dedicated Instagram and Facebook profile APIs are documented; this is not the same four-platform set.Not established in reviewed docs. No dedicated platform endpoints identified in the reviewed Search API docs.Not established in reviewed docs. The reviewed search docs do not establish dedicated social-platform endpoints.
    AI answer + source linksShips. AI Search returns an optional summary and result source links in one response; inline citations are not promised.Ships. Search can return an optional answer with result URLs in one response; Research also offers formatted citations.Ships. Answer API returns generated answers and citations.Partial. Parses Google's AI Overview and references; it does not generate the answer.Ships. Agent API returns generated answers with source citations.Partial. Agent can return structured findings with source URLs when the output schema requests them; this is not a standard cited Q&A response.
    Extract a supplied URLShips. /web/extract accepts a URL and returns extracted text or HTML.Ships. Extract API accepts URLs and returns page content.Ships. Contents API retrieves page text from supplied URLs.Not established in reviewed docs. Search-result parsing does not establish a general endpoint for extracting an arbitrary supplied URL.Not established in reviewed docs. Search returns content for its results; the reviewed docs do not establish arbitrary URL extraction.Ships. Scrape API extracts a supplied URL.
    Crawl linked site pagesNot established in reviewed docs. /web/crawl is a deprecated alias for single-URL /web/extract; a linked-site crawler is not documented here.Ships. Crawl API follows links from a starting URL.Partial. Contents can fetch a bounded set of linked subpages; it does not enumerate an entire site.Not established in reviewed docs. The reviewed Search API docs do not establish linked-site crawling.Not established in reviewed docs. The reviewed Search API docs do not establish linked-site crawling.Ships. Crawl API follows links across a site.
    MCP serverShips. Official MCP integration.Ships. Remote MCP server.Ships. Exa MCP server.Ships. Official MCP integration.Ships. Official MCP integration.Ships. Hosted MCP server; the keyless setup exposes Search, Scrape and Parse.
    Cost or usage in responseShips. Successful billable responses expose exact USD cost in the body or X-Desearch-Cost-Usd header.Partial. Search can include credits used when include_usage is enabled; no per-call USD amount is shown.Ships. Search and Contents responses expose estimated costDollars; billing uses usage counters.Not established in reviewed docs. A per-request cost or credit field is not established in the reviewed Search API response docs.Partial. Agent and Sonar responses show usage.cost.total_cost; this is not established for the raw Search API.Partial. Agent job status shows creditsUsed; this is not a universal per-request USD cost field.

    Questions

    Benchmark FAQ

    What do these benchmark scores measure?
    SimpleQA Verified and SealQA report whether a benchmark reference page appears among the top results. FRAMES reports the share of required reference pages found in the top 10. BrowseComp grades the final answer of an agent using each provider's search. These measures should not be averaged into one score.
    Does finding a reference page mean the search result answers the question?
    No. The published SimpleQA Verified and SealQA scoreboards also contain separate measures for whether a fetched page or the returned text states the verified answer. This page displays those results alongside reference-page retrieval.
    Are the BrowseComp provider settings identical?
    No. In this published run, the Desearch profile allowed up to 20 results per search and the other profiles up to 10. The same agent model and 100 sampled questions were used, but the result-count difference limits a direct provider ranking.
    Are these independent third-party results?
    No. Desearch ran these evaluations. The runner code, published scoreboards, and per-question results are available for inspection through the linked repository and dataset.