NewsReach: Recommender Distribution Audits for Publishers

Concept

A vertical analytics product answering one question for news organizations: “how does our content actually get surfaced?” A consented, demographically diverse panel installs the monitoring extension; the service measures, per platform (search engines, social feeds, aggregators), how often and where the publisher's stories appear to different participant segments — reach by segment, ranking position, and divergence between segments. The single deliverable is a quarterly distribution report that platforms' own dashboards and APIs cannot provide: evidence of algorithmic distribution of your content across real, consented users. Publishers use it to ground platform negotiations, public-interest reporting, and regulatory submissions about visibility.

Unlike general-purpose auditing services that let researchers or regulators run ad-hoc studies, this is a fixed-scope, publisher-specific product: the unit of analysis is one publisher's stories across a standing panel, delivered as a recurring quarterly report.

How the panel is recruited and kept representative

The panel is the main operational hurdle and cost driver, so the design is explicit about it:

• Seeding through publishers' own audiences. Participating publishers recruit the first participants through channels they already own and are trusted in — on-air and on-site calls to action, newsletters, apps, membership programs. Public broadcasters are the best fit because their statutory remit gives them both the motivation and the audience relationship.
• Quota-filling for representativeness. Recruiting only from a publisher's own audience would skew the panel toward loyal, heavier news consumers. The service therefore sets demographic quotas (age, region, education, income band) matched to the population the publisher serves, and fills the gaps through a commercial research-panel vendor or targeted paid outreach.
• Recurring incentives, not one-off rewards. Participants receive a modest recurring payment each quarter for keeping the extension installed and active, because the value of a participant is sustained coverage, not a single data point. Payment is for participation itself, never for producing any particular finding.
• Quarterly re-balancing. Panel composition is measured every cycle, attrited segments are replaced, and the panel is over-recruited to buffer churn. Each report discloses the panel's actual composition and its limits, so buyers can see which population the audit speaks for.

The cost model assumes a blended cost per participant: cheap seeding through publisher channels, expensive quota-filling through vendors. The real per-participant cost — and therefore the price of a quarterly report — stays an open question until a pilot measures it.

Why now

Paper [0] demonstrates that participant-side monitoring across search engines, social media, and political platforms is technically feasible within GDPR (white lists, pseudonymization, rolling deletion 5 years after last publication). The data needed to measure news distribution — what search results and social feeds show each participant — is the same class of data such monitoring collects, so the audit applies an already-demonstrated method to a new question rather than inventing new collection infrastructure. The recommender survey [1] documents bias and unfairness in what recommenders surface, and the Council of Europe report [2] establishes that unequal algorithmic visibility is a discrimination-class concern — so both publishers and regulators need segment-level visibility evidence. Public broadcasters with statutory remits to reach all of society are the natural first buyers, and the paper's Dutch pilot context makes European public media the obvious launch market.

Open questions

• What is the actual cost per participant for the hybrid recruitment model above (publisher-channel seeding plus paid quota-filling), and can a quarterly report be priced within what publishers will pay?
• Do current browser extension permissions and platform Terms of Service allow for the systematic scraping of feed rankings without triggering account bans for participants?
• To what extent does the 'observer effect' (participants changing behavior because they are being monitored) skew the recommendation algorithms being audited?
• Will platforms like Google or Meta legally challenge the collection of ranking data as a violation of their proprietary API or Terms of Service, even if users consent?
• Is there a documented willingness among public broadcasters to pay for 'evidence of invisibility' if it cannot be directly converted into increased traffic?
• How fast does panel attrition run, and does quarterly re-balancing keep the composition stable enough for quarter-over-quarter comparisons?