The week's news,
grouped into stories.

A scraper agent fetches articles from every source configured for a topic — both RSS / Atom feeds and ordinary HTML pages. For an HTML source it discovers the article links on a listing page, then pulls each full article body with trafilatura (falling back to BeautifulSoup) rather than trusting feed snippets.

• Each article is filtered by keyword and by semantic relevance to the topic, using a configurable cosine-similarity threshold.
• Duplicates are dropped by URL and title similarity before storage.
• Survivors land in PostgreSQL with their extracted text, ready for the weekly pass.

The core step. Each week the last seven days of articles are embedded and grouped into stories.

• Embeddings: a sentence-transformer model running on an ONNX backend. Each article is embedded as a hybrid of its title plus the start of its body — a richer signal than the title alone.
• Similarity: cosine similarity between article embeddings.
• Clustering: configurable per topic — DBSCAN, HDBSCAN, or greedy — with a tunable similarity threshold.
• Geographical penalty: spaCy named-entity recognition extracts the locations each article mentions; when two articles name different, non-overlapping places their similarity is reduced — so a factory fire in Hamburg and one in Osaka never fuse into a single story.
• Quality gate: a cluster only becomes a top story if it draws on at least two independent sources; single-outlet clusters are dropped as noise.

The top N clusters move on, each sampled for central, recent, and deliberately diverse articles so the summary sees the full spread.

Each story is summarized from the perspective of every source that covered it, using a large language model — a cloud model, or a local Ollama model, configurable.

• Two-pass: an intermediate summary is built from the cluster, then condensed into the final — steadier on large clusters.
• LLM-as-judge: a separate judge agent reviews each summary for hallucinations and rejects ones that don't hold up.
• Name validation: named entities in a summary are checked against the source articles; anything that was never in the source is flagged as a likely hallucination.

Every article's tone is scored, then averaged per source within each story.

• Primary model: pysentimiento (transformer-based, multilingual), with VADER and TextBlob as fallbacks.
• Each article gets a compound score from -1.0 (very negative) to +1.0 (very positive), labelled negative / neutral / positive.
• Aggregating by source shows, at a glance, which outlets framed a story warmly and which didn't.

The finished analysis is rendered to HTML, Markdown and plain text with Jinja2, themed from the topic's palette (the same colours dressing this page).

• Email CSS is inlined with css_inline for client compatibility, with dedicated dark-mode handling.
• Each subscriber gets a personalized email carrying an HMAC-signed unsubscribe / manage-subscriptions token.
• Delivery runs through a pluggable provider layer — SMTP, Resend, or EmailJS. Every report is archived in the web app.