Every hour, Signalwaves reads around 8,000 articles from 500+ sources, groups them into stories, and shows how differently source camps frame the same event. This page explains the architecture behind it.
The pipeline and the interface never talk to each other while someone reads. A Python pipeline writes its results as static files every hour; the web app and the store apps only serve and read those files. A failed pipeline run freezes the data for an hour. The app stays up.
A scheduled job starts the cycle every hour. Segment widths show the real duration of a run.
500 RSS feeds in parallel, with timeouts per feed. A feed that keeps failing is skipped for the run.
Platform, aggregator and community APIs, queried with the day's biggest stories so coverage grows around them.
Niche lenses are refilled; every link gets outlet region, language, media kind and reader emotion.
Snapshot for the interface, hourly archive, and the unfiltered firehose of all articles.
A separate step clusters the last three days into stories and sorts them into lenses.
Stories are rebuilt every hour from fresh coverage instead of being kept as long-lived topics.
Take every article from the last three days.
Embed each headline with a multilingual sentence model, so a German, Spanish and English article on the same event land together.
Cluster greedily, newest first: an article joins a story when it is close enough to the story's centre.
Keep only events covered by at least two independent outlets.
Carry a story's identity across hours while it develops; end it after seven days so names can't go stale.
Name it after the headline closest to the centre, preferring primary sources.
Sort it into 60+ thematic lenses by topic and by fit to each lens profile.
Every classification is heuristic and disclosed as such. No editorial judgement is involved.
| Signal | Values | How |
|---|---|---|
| Source tier | Official · Wire · Editorial · Platform · Aggregator · Community | How a source works, corrected by a curated domain list. |
| Outlet region | 12 world regions | Where the outlet is published, not what the article is about. |
| Language | ISO language code | Script detection, then a statistical detector. Headlines appear in your language first. |
| Media kind | Mainstream · Independent | Curated domain lists; platforms stay unclassified. |
| Reader emotion | Concerning · Hopeful · Curious · Outrageous | A lexicon first, a transformer model for ambiguous cases. |
| Divergence | Consensus · Mixed · Contested | How far source camps pull apart in emotional framing; contested needs at least four sources. |
The most contested stories of the day, the broad consensus, and today's mood compared with a normal day.
Every story, sortable by divergence, with a viewpoint matrix that lays the camps side by side.
The unfiltered firehose, strictly by time, filterable by tier, region, emotion and media kind.
Thematic perspectives from geopolitics to niche sports; favourites follow you across the app.
The store apps load the live web app, so every improvement reaches all platforms at once.
Many camps, tiers and regions make the pool resistant to single-publisher bias.
A dead feed, a missing key or an unavailable model never stops the rest of the run.
Tiers, emotions and regions are algorithmic and labelled that way.
Preferences stay on the device. Analytics only with consent.