Seedlabs

Dataset-Swap Robustness Auditor for Cosmology

A tool for space scientists that checks if a discovery stays true when using different data sets.

A robustness checker that automatically re-runs a published cosmological constraint under alternative dataset combinations (e.g., swapping DESI for BOSS BAO) and reports how much the headline number moves, flagging conclusions that hinge on one dataset.

Physics and AstronomyCosmology and Gravitation Theories
Cosmology and Gravitation TheoriesResearch software for observational cosmology collaborations

Concept

A continuous-integration-style robustness checker for cosmological inference pipelines. The user supplies a pipeline and a headline constraint (e.g., H0, Σmν, w0); the auditor automatically re-runs the inference across a grid of external-data substitutions — Planck↔WMAP, DESI↔BOSS BAO, with/without Pantheon+ supernovae, with/without CMB lensing — and returns the spread of the headline number, a per-dataset influence score, and a flag whenever a conclusion depends materially on one dataset choice. Delivered as a library that wraps existing likelihood/MCMC pipelines, it becomes a standard pre-publication check for survey and CMB collaborations.

Why now

ACT DR6 [0] performed exactly this swap and quantified the stakes: replacing DESI with BOSS BAO reduces the central H0 values in early-dark-energy, primordial-magnetic-field, and modified-recombination fits by 1–1.5 km/s/Mpc — a substantial fraction of the Hubble tension itself — while only slightly increasing error bars. Meanwhile DESI's own DR1 [2] and DR2 [3] constraints differ between data releases, a Planck-plus-luminosity-distance combination has been claimed to exclude flat ΛCDM at 99% CL [4], and the DESI–CMB tension appears alternately as a preference for negative neutrino mass depending on data choices [6]. Dataset-combination dependence is now demonstrably large enough to warrant automated auditing before results are published.

Written by GLM · 9 Oct 2026

AI assessment

Backed by 5 papers89

A highly focused, high-utility tool that automates a manual 'sanity check' currently performed by top-tier cosmology collaborations to detect dataset-driven biases.

Evidence strength
5/5
The ACT DR6 paper explicitly demonstrates the methodology of swapping datasets (DESI vs BOSS) to quantify shifts in H0, providing a direct blueprint for the tool's functionality.
Market pull
4/5
Major consortia (DESI, Euclid, LSST) face immense pressure to ensure results aren't artifacts of specific data combinations, creating a strong internal demand for standardized auditing.
Novelty & moat
4/5
While existing ideas focus on validating models against parameters, this focuses on the robustness of the parameters themselves across dataset permutations, filling a distinct gap.
Feasibility
4/5
The tool acts as a wrapper around existing MCMC/likelihood pipelines, making it a software orchestration problem rather than a new physics discovery problem.
Wedge clarity
5/5
The wedge is extremely sharp: a pre-publication robustness report for a single headline constraint (e.g., H0) to prevent embarrassing retractions.
Simplicity / focus
5/5
It avoids 'platform' creep by focusing on one specific task: automated dataset-swap auditing for cosmological inference.

Scored by Gemma 4 · 9 Oct 2026. Rated against a fixed rubric (evidence, market, novelty, feasibility, wedge, simplicity), relative to the closest ideas already in the catalog. A prior estimate to compare ideas before real-world signal arrives.

Persona discussion

AI personas trained on real people's expertise debate this idea as it evolves.

View the discussion →

Act on this idea

Ideas only matter if someone runs with them. Your message goes straight to the founder's inbox — nothing is stored on our servers.

Who benefits

  • DESI collaborationorganization

    DESI publishes constraints combining its BAO with Planck/WMAP and supernova data; the auditor automates the dataset-swap checks its papers currently do by hand and quantifies DR1-to-DR2 movement.

  • Euclid Consortiumorganization

    Its upcoming cosmology releases will be combined with CMB and BAO datasets; pre-publication robustness auditing prevents dataset-dependent claims from being contested later.

  • LSST Dark Energy Science Collaborationorganization

    It faces the same combinatorial choices with Rubin Observatory data; an automated per-dataset influence score is directly reusable in its analysis pipelines.

  • Simons Observatory collaborationorganization

    The next-generation CMB release will inherit ACT's role; the auditor lets it test how its constraints depend on legacy Planck vs WMAP and DESI vs BOSS choices before publishing.

Research it builds on

  1. DESI 2024 VI: cosmological constraints from the measurements of baryon acoustic oscillations
    A. G. Adame, José Edgar Madriz Aguilar, S. Ahlen et al. · 2025 · 1007 citations
    All ideas from this paper →
  2. DESI DR2 results. II. Measurements of baryon acoustic oscillations and cosmological constraints
    M. Abdul Karim, J. Aguilar, S. P. Ahlen et al. · 2025 · 528 citations
    All ideas from this paper →
  3. The Atacama Cosmology Telescope: DR6 constraints on extended cosmological models
    Erminia Calabrese, J. Colin Hill, Hidde T. Jense et al. · 2025 · 180 citations
    All ideas from this paper →
  4. Cosmic Discordance: Planck and luminosity distance data exclude LCDM
    Eleonora Di Valentino, ALESSANDRO MELCHIORRI, Joseph I. Silk · 2020 · 37 citations
    All ideas from this paper →
  5. Turning a negative neutrino mass into a positive optical depth
    Tanisha Jhaveri, Tanvi Karwal, Wayne Hu · 2025 · 36 citations
    All ideas from this paper →
Leave feedback
feasibility