How Retailopedia analyzes signals

Retailopedia continuously crawls Indian retail sources, enriches each item through a multi-agent AI council running on a frontier reasoning model, and publishes only the signals that survive a devil’s-advocate review and a numerical validator. Every published signal is named, dated, cited to a primary URL, and paired with a counterview. No human fingers touch the pipeline after intake — but every agent’s output is stored, so a reviewer can audit why we wrote what we wrote.

What feeds the system

Retailopedia tracks 52 Indian retail publishers today, with the list growing as we add beats. Roughly half are RSS-parseable; the rest are JavaScript-rendered list pages that we fetch through a stealthed headless browser with per-source timeouts and circuit breakers. Each source has a declared cadence, a URL-hash dedup, and a per-cycle budget so one slow publisher cannot starve the rest.

The current set includes the retail-trade press (The Hindu BusinessLine, ET Retail, ET Realty · Shopping Malls, Retail4Growth, IMAGES Business of Fashion, Indian Retailer, India Retailing, BW Retail World, Storyboard18), the generalist business desks that file retail stories (Mint, Financial Express, Business Standard, Business Today, Forbes India, Fortune India, Moneycontrol, CNBC TV18, The Economic Times), the startup press (Inc42, Entrackr, YourStory, The Ken free list), and regulatory feeds like BSE Corporate Announcements for listed-retailer filings. The full source list — with the number of signals filed from each publisher — lives on /trust, updated live from the database.

We do not ingest paywalled article bodies. Where a source is paywalled, we read the public excerpt and the headline, cite the publisher, and link out.

The editorial council

A council run is sequential by stage, parallel within a stage, and entirely non-interactive. Each agent’s output is a structured JSON object stored against the signal, so if the Chief signs off on a bad signal we can walk back the exact agent whose read was off.

SCOUT — extracts structured facts from the raw wire item: primary brand, secondary brands, cities mentioned, event type, numeric claims, time horizon, and a 45-word neutral summary. This is the gate for retail-India relevance. Non-retail items die here and never cost a downstream LLM call.

CLASSIFIER — scores each scouted item against a standing editorial brief: is this novel, is it substantive, is it well-sourced. Produces a 0-100 worthiness score. Items below the bar are dropped before the Editor writes; this is where most noise is eliminated.

EDITOR — writes the signal the reader actually sees: a primary theme, a typed headline ≤140 chars, a deck ≤400 chars, a delta one-liner, and a confidence score. The Editor is constrained to one of a fixed taxonomy of themes (store-opening, retail-brand, omni-channel, retail-company, footfall, leadership) so the signal feeds the right topic hub.

FORECASTER — projects 2nd-order effects. For each signal it produces a probability-weighted scenario tree (typically three branches), a set of next moves to expect, and triggers to watch. The forecast is explicitly a scenario with a probability, not a prediction.

COMPETITOR-MAPPER — identifies the 2-4 rival Indian brands most affected by the signal, with a 0-10 threat score and a recommended counter-move per rival. Writes edges into the competitor graph (competes_with relationships), which is what powers the related-brands rails on /brand/amul, /brand/ixigo, and every other brand hub.

DEVIL’S ADVOCATE — writes the strongest counter-read against the signal: the bear case, the missing context, who would disagree, and a 0-10 strength score. This is the non-negotiable agent. Every signal that ships carries its own counterview on the signal page, sourced directly from this agent’s output. A signal with a weak counterview is a signal we are probably wrong about.

VALIDATOR — the numerical fact-checker. Given the source body and the Editor’s signal, it pulls every number from the signal and asks whether the source supports it. Numbers that fail validation get struck; signals that lose a structurally-necessary number get demoted or killed.

TREND CONTEXT — grounds the signal against recent history for the same theme and brand. It reads the last 30-90 days of filed signals and writes a one-paragraph situational read — is this an escalation, a reversal, or business-as-usual.

BRAND-LENS — writes three one-sentence takeaways for three personas (operator, investor, corporate-development). Persona surfaces on the signal page come from here.

LIBRARIAN — runs across the day’s finalised signals and cross-links them to existing brand, theme, and city entities. Updates the entity graph with fresh mention counts and last-seen timestamps.

QUANT — computes per-cycle aggregate metrics: brand frequency, theme mix, city activity, and a short trend-of-the-day note. Drives the counters on the homepage and the velocity chart on theme pages.

CROSS-REF / ARCS — looks across a cycle’s finalised signals for narrative arcs — three or four signals that together tell a larger story. Produces arc records that feed the arcs block on the homepage and thread signals across dates.

COUNCIL DEBATE — a moderator prompt that holds a short debate between the Editor, Classifier, and Cross-Ref reads to promote the strongest signals and demote the weakest. The debate output is stored, not summarised away.

CHIEF — the final editorial close-out. Reads the cycle’s promoted signals and the debate output, writes the day’s wrap, and signs off on the publish set. Dissent from earlier agents is preserved in the stored record.

Debate and consensus

Disagreement is not smoothed over. When the Devil’s Advocate writes a strong bear case, it does not vanish — it becomes the counterview on the signal page. When the Validator strikes a number, the Editor rewrites or the signal gets demoted. The Debate moderator explicitly promotes and demotes within the finalised set before the Chief signs off, and the promote/demote record is stored.

Every signal publishes with a confidence score (0-1) that reflects the council’s converged read: Editor’s confidence, weighted by Validator pass-rate and Devil’s Advocate strength. The score surfaces on the signal page, not just as chrome — low-confidence signals are visible as such, and a reader who cares can sort by it. Signals that fall below the internal confidence floor are not published at all; they remain in the database as a rejected record so a later audit can ask why they were killed.

Data freshness

The architecture is built around the principle that writes should not trigger site builds. The council writes a new signal straight to the primary datastore, which fans out to an internal publish bus. Readers on /live hold an open Server-Sent Events connection to that bus and receive the new signal within seconds of council completion. There is no static build in the hot path — /live is server-rendered on each request, and signal detail pages like /signal/swiggy-seeks-shareholder-nod-to-become-iocc-paving-way-for-i-cad35c79 read from the same database, so a correction takes effect on the next request.

On publish we also fan out to discovery surfaces: the sitemap-signals.xml entry is inserted in the same database transaction as the signal row, the RSS feed regenerates on read, and an IndexNow ping notifies Bing and Yandex that the URL is new. Google’s crawler picks up the sitemap on its own schedule.

Correction policy

If a signal is wrong, email [email protected]. We investigate within 48 hours. A confirmed correction updates the signal’s modification timestamp, which surfaces on the signal page as a visible Updated: line and propagates into the schema.org JSON-LD as dateModified. For substantive changes — the wrong brand, the wrong number, the wrong event — we do not silently edit. The old signal is marked superseded and linked to its replacement, so citations to the original URL still resolve and a reader can see the prior reading and the reason it was replaced. The supersede pair stays in the history forever. The corrections log is summarised on /trust.

What we don’t do

We don’t have browsing agents synthesising novel takes from the open web. Every signal traces back to a specific cited source row in the ingest archive.

We don’t paraphrase paywalled articles. We summarise the public excerpt and link to the publisher. If a source moves fully behind a paywall we drop it.

We don’t A/B test against user behaviour. The signals you see are the signals a reviewer would see. There is no personalisation engine deciding what to show you first; the ranking is editorial, not behavioural.

We don’t publish unsolicited digests. There is no email list that arrives in your inbox uninvited. You either come to /live or you don’t.

We don’t claim to predict the future with certainty. Every forecast the Forecaster writes is a scenario with an explicit probability and a horizon. When we’re wrong about a scenario, the scenario stays in the record.

The live feed: /live. The full source list, controls, subprocessors, and corrections address: /trust. What retail intelligence is, more broadly: /about and the companion pillar what is retail intelligence. The council’s artifacts are most legible on the signal pages themselves — pick any published item from the feed, like /signal/india-gold-discounts-hit-record-207-oz-as-15-duty-hike-empti-2fab8c52, and the scout summary, forecast, counterview, and cited source are all on one page.