Embedding tasks are part of SN22 main: GPU miners turn the pages the network crawls into vectors with Desearch-Embedding-4B, validators recompute a sample to check the work, and that verified work is what makes the Desearch index searchable.
If you already know the model and the crawl and embed loop from Desearch 2.0, this post is the practical part. It covers what an embed task is, what you need, the two steps to join, and how checks and rewards work. Everything here comes from the public subnet-22 PR #391.
How a crawled page becomes searchable
Crawl miners fetch pages. Embed miners turn the text of those pages into vectors. Validators embed a sample again and compare. Vectors that hold up are what the search index is built from. Crawling gives the network pages; embedding is what lets a query find the right ones. For background on how Desearch builds a web index on Bittensor, start there.
What an embed task is
A task is a batch of crawled page texts. You run the model on your own GPU and send back one vector per text, in the same order. Validators pick a sample of your uploads, embed the same texts again and check that the results match yours.
What you need
- An NVIDIA GPU with 16 GB of memory or more. 24 GB is recommended.
- vLLM 0.31.0.
- A registered hotkey on SN22. If you have not set one up, the miner developer guide walks through it.
- The open model, Desearch-Embedding-4B on Hugging Face. No Hugging Face account is needed.
Step 1: Start the model server
From the PR #391 instructions:
vllm serve desearch/Desearch-Embedding-4B --revision 12e5c6910d49dd5534e8a77a6fa7d8a057bd534a \
--served-model-name desearch-4b-v1 --runner pooling --dtype float16 \
--no-enable-chunked-prefill --max-num-batched-tokens 16384 --port 8000Run it exactly as given. A different precision, quantized weights or another revision give different vectors, and those fail the check.
Step 2: Start the embed miner
pm2 start python3 --name desearch_embed_miner -- -m neurons.miners.embedSettings are in neurons/miners/.env:
EMBED_MODEL, by defaultdesearch-4b-v1EMBED_API_URL, your model server, by defaulthttp://127.0.0.1:8000/v1/embeddingsEMBED_API_KEY, only if your server needs one
The miner waits for the server before it claims any task.
Already crawling? Nothing changes
Crawl miners need no changes. One hotkey can crawl, embed or both. Run the embed miner next to your crawl miner under the same hotkey, or on its own. Crawling and embedding keep separate budgets and lockouts, so a problem in one does not touch the other.
How the check and rewards work
Validators check a sample of each miner's uploads. A passing task, and every task not picked for a check, is paid by the characters it embedded. A failed check earns nothing and takes back what you were paid since your last passed check. So the simplest way to earn steadily is to run the server exactly as given and keep it healthy.
On emission, PR #391 puts it this way: "50% is burned, 20% goes to crawl miners and 30% to embed miners." For more on how SN22 rewards work in general, see how developers earn on Desearch and the scoring and incentives guide.
For validators
Validators update themselves. To check embed work, a validator either runs the same model server on its own GPU and sets EMBED_API_URL in neurons/validators/.env, or falls back to Desearch's temporary model server through the task API. Either way the check is the same.
FAQ
Do crawl miners need to change anything?
No. Crawl miners need no changes.
Can one hotkey crawl and embed?
Yes. One hotkey can crawl, embed or both, and each keeps its own budget.
What GPU do I need?
An NVIDIA GPU with 16 GB of memory or more. 24 GB is recommended.
Which model do embed miners run?
Desearch-Embedding-4B, open on Hugging Face.
How are embed miners rewarded?
By the characters they embed, for tasks that pass a validator check or are not picked for one. A failed check takes back pay since the last passed check.
What if a validator has no GPU?
It falls back to Desearch's temporary model server through the task API.
Search the index SN22 builds
The pages miners crawl and embed feed the index behind Desearch. You can run AI Search on web and X from the Desearch Console today.