Seedlabs

Frontier Judgment Assessment: Testing Who Knows When to Distrust AI

A test that measures whether an employee knows when to trust or distrust an AI's answer.

A 90-minute assessment that blind-grades each employee's work with and without AI — including tasks where AI makes people wrong — and scores their ability to detect when to distrust AI, giving L&D and staffing leads a per-person 'AI judgment' profile.

Computer ScienceArtificial Intelligence Applications
AI Governance and Cognitive ImpactCorporate learning & development and talent assessment in professional services

Concept

A ~90-minute per-person assessment, sold to firms deploying AI, that scores each knowledge worker's frontier judgment — the ability to tell when AI is helping and when it is hurting. Each participant works through a calibrated task mix: several inside-frontier tasks (creative and analytical, like the 18 in the field experiment) with AI access, one or more outside-frontier tasks (complex, multi-constraint managerial judgment) with AI access, and one no-AI baseline task. Outputs are blind-graded. The scorecard reports (a) the worker's quality delta with versus without AI, and (b) whether they verified, questioned, or overrode AI output on the outside-frontier task. The deliverable is a per-worker profile plus a cohort heatmap that L&D and staffing leads use to decide who can work with AI autonomy and who needs a verification step or targeted coaching.

Why now

[0] demonstrates that the inside-frontier gain and the outside-frontier loss happen to the same people: 758 consultants gained on all 18 inside-frontier tasks (12.2% more tasks, 25.1% faster, significantly better quality) yet were 19% less likely to be correct on the outside-frontier task — and because the boundary is jagged and invisible, individual judgment about when to distrust AI varies and is worth measuring. Firms currently invest in prompt training (the experiment's third condition was a prompt-engineering overview), but no instrument exists for testing judgment about when to distrust AI. [2]'s scoping review confirms that generative AI use raises documented concerns about critical engagement and accuracy of generated content, making critical engagement — not prompting — the assessable skill firms currently have no visibility into.

Written by GLM · 9 Oct 2026

AI assessment

Backed by 2 papers86

A high-utility diagnostic tool that converts a specific academic finding on the 'jagged frontier' into a tangible HR assessment for professional services firms.

Evidence strength
5/5
The idea directly operationalizes the methodology and findings of the BCG field experiment [1], which explicitly demonstrated the coexistence of inside- and outside-frontier performance shifts.
Market pull
4/5
High-margin professional services firms (BCG, McKinsey, etc.) have a strong incentive to mitigate the risk of 'invisible' AI errors in client deliverables.
Novelty & moat
3/5
While it complements 'Frontier Pilot' and 'Rumble Strip', it shifts the focus from task-mapping and real-time warnings to individual human competency assessment.
Feasibility
4/5
The MVP requires a curated set of tasks and a grading rubric, which is straightforward to build given the existing research parameters.
Wedge clarity
5/5
The 90-minute 'AI Judgment' assessment is a sharp, discrete product with a clear value proposition for L&D leads.
Simplicity / focus
5/5
The product is a single, focused diagnostic tool rather than an over-scoped platform.

Scored by Gemma 4 · 9 Oct 2026. Rated against a fixed rubric (evidence, market, novelty, feasibility, wedge, simplicity), relative to the closest ideas already in the catalog. A prior estimate to compare ideas before real-world signal arrives.

Persona discussion

AI personas trained on real people's expertise debate this idea as it evolves.

View the discussion →

Act on this idea

Ideas only matter if someone runs with them. Your message goes straight to the founder's inbox — nothing is stored on our servers.

Who benefits

  • Boston Consulting Groupcompany

    Can identify which of its consultants can be given AI autonomy and which need verification steps, directly extending what the experiment revealed about its own workforce.

  • McKinsey & Companycompany

    Would use per-worker judgment profiles for staffing and L&D targeting as it scales internal AI tools across its consultant base.

  • Deloittecompany

    Runs large-scale campus and L&D programs; a judgment assessment gives it a concrete instrument for certifying AI readiness of tens of thousands of client-facing staff.

  • Harvard Business Schoolorganization

    Funded the underlying field experiment and teaches MBAs entering AI-augmented consulting; the assessment is a natural teaching and research instrument for AI-era judgment.

  • Baker McKenziecompany

    A global law firm deploying legal AI to junior lawyers; knowing which staff over-trust AI output informs supervision and training decisions.

Research it builds on

  1. Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of Artificial Intelligence on Knowledge Worker Productivity and Quality
    Fabrizio Dell’Acqua, Edward McFowland, Ethan Mollick et al. · 2026 · 170 citations
    All ideas from this paper →
  2. Enhancing Critical Thinking Skills in ChatGPT-Human Interaction: A Scoping Review
    Lilia Raitskaya, Елена Викторовна Тихонова · 2025 · 27 citations
    All ideas from this paper →
Leave feedback
feasibility