Build a topic RSS feed generator in Python

This RSS feed generator turns topic search results into a feed with Python's standard library. Developers can write an XML file or serve it locally. This page shows a stored sample; there is no hosted demo.

The problem

A topic can span many sites without a single RSS subscription. Repeating searches leaves you sorting links by hand. A feed reader needs dated items with stable links and descriptions.

What the app does

• Searches one topic within a date window using Desearch.
• Drops undated pages and deduplicates normalized URLs and titles.
• Sorts dated pages newest first and writes RSS 2.0.
• Uses the first highlight as the description, capped at 500 characters.
• Caches result JSON for six hours and offers a localhost feed route.

How it works

1. Set the topic and window. The default window runs from seven days ago through today. Count can be 10, 15 or 20.
2. Search or reuse results. POST /search uses balanced mode and highlights. The cache key includes every request parameter and the date window. --no-cache bypasses cache reads and writes.
3. Clean the items. The script recovers original URLs from translation links where possible. It removes URL fragments, trailing slashes and tracking parameters starting with utm_. It deduplicates URLs and normalized titles before sorting.
4. Write the feed. Each item contains a title, link, URL-based guid, publication date and description. Dates have day precision and become midnight UTC in RSS.
5. Read locally. The CLI writes a file or prints XML. The optional server binds to 127.0.0.1 and returns RSS at /feed?topic=....

These are dated search results, not an exhaustive news feed. Evergreen pages can appear. Syndicated stories with different URLs and titles may remain separate items.

The code

This verbatim excerpt from topic_rss.py builds the request body. Imports and validate_topic are defined in the full file. The client reads DESEARCH_API_KEY from the environment and sends it directly in the Authorization header.

def build_request_body(topic, days=7, count=20, today=None):
    topic = validate_topic(topic)
    if days < 1:
        raise ValueError("days must be positive")
    if count not in (10, 15, 20):
        raise ValueError("count must be 10, 15, or 20")
    today = today or date.today()
    return {"query": topic, "mode": "balanced", "count": count,
            "start_date": (today - timedelta(days=days)).isoformat(),
            "end_date": today.isoformat(), "highlights": True}

Endpoints

• POST /search

Example output

This single item is a verbatim excerpt from sample-feed.xml. That stored sample uses a checked-in fixture and a fixed build date. It illustrates the feed format rather than a new live search.

    <item>
      <title>Bittensor overview</title>
      <link>https://example.org/bittensor/overview</link>
      <guid isPermaLink="true">https://example.org/bittensor/overview</guid>
      <pubDate>Thu, 22 Oct 2026 00:00:00 +0000</pubDate>
      <description>An overview of Bittensor, its subnets, and the roles of network participants.</description>
    </item>

Run it yourself

Get a key from the Desearch console and set DESEARCH_API_KEY in your environment. Open the source folder and use Python 3.10 or newer. No packages are required.

These commands are copied verbatim from the source README:

cd topic-to-rss
python3 topic_rss.py 'bittensor' -o feed.xml
python3 topic_rss.py 'bittensor' --days 7 --count 20
python3 topic_rss.py 'bittensor' -o feed.xml --no-cache

To use a feed reader, start the local server with the source README's commands:

python3 topic_rss.py --serve --port 8000
curl 'http://127.0.0.1:8000/feed?topic=bittensor'

What it costs

One uncached refresh with count 20 cost $0.01 on 2026-10-23. The same recorded run reported 0.568 seconds of fetch latency. This is one measurement, not a speed guarantee. The script prints cost_usd, latency_s and cache_hit to stderr.

A fresh cache hit reuses stored results without an API call. Changing parameters or crossing a date boundary can trigger a new request. Simultaneous cache misses can each send a request. Check the returned cost and current pricing before scheduling refreshes.

Ideas to extend it

• Add an allowlist of source domains for a narrower feed.
• Group similar titles to reduce syndicated copies.
• Keep a history of seen URLs across refreshes.
• Add separate subscriptions for several topics.