Desearch x Bittensor
How Desearch runs on Bittensor Subnet 22, and what that means for you as an API user.
How it works
Subnet 22 builds Desearch's web index. Pages worth keeping are found, miners crawl them and turn their text into embeddings, validators check a sample of both, and passed work goes into the collection the search index is built from.
- Crawl miners claim crawl tasks, fetch every assigned page through their own proxies, and extract the text (title, dates, headings, main text) plus the page HTML. They upload the result and report the task complete. They need no open port. Miners do not serve search queries, and queries never wait on miner work.
- Embed miners claim embed tasks and turn the text of crawled pages into embeddings with Desearch's embedding model, on their own GPU. The same miner can crawl and embed.
- Validators randomly check part of each miner's pages by fetching them themselves and comparing the text, and recompute a sample of each miner's embeddings to check that they match. They report pass or fail. A crawl task fails only when the mismatch is significant. Validators set weights from the record of every miner's uploads.
- A miner's reward follows its accepted work. Crawl miners are paid for accepted pages and embed miners for accepted embeddings, each compared with every other miner doing the same task over a recent window. A failed check takes back what was paid since the miner's last passed check. A task score is not a fixed token payment, and no role has guaranteed rewards.
Rounds are fixed in advance, actions are signed, and uploads go straight to storage. Which work gets checked is decided by chance after upload, and results are public.
Desearch runs the search API on top of the index.
What you need to know
Nothing about Bittensor changes how you integrate. You authenticate with an API key, pay in USD, and never touch the TAO token or run any network software. Developers use the Desearch API via Console, and they do not need to run a miner.
📘 Read more about Bittensor
Please visit https://docs.bittensor.com/ to see how Bittensor operates.