The Guidance Design Kit: Worksheets for Building Decision-Support into Analytics Tools

Concept

A digital, facilitator-led worksheet kit for teams that build analytics products. In a half-day session, a product/UX team: (1) maps the decision points their users hit (which algorithm, which chart type, which data subset, which parameter value — and, under uncertainty, which option or plan to commit to); (2) for each decision point, derives the realistic alternatives and defines the evaluation criteria; (3) specifies the comparison visualization that will present those alternatives to end users; and (4) runs the new Uncertainty Guidance Pattern Module — a pattern library and worksheet that classifies candidate guidance along two validated dimensions from the proton therapy planning study: degree of intrusiveness (from ambient, passive indicators to directed, interruptive prompts) and detail-orientation (from summary-level cues to full uncertainty detail). Teams plot candidate patterns for each decision point on this grid, choose a starting pattern per user segment, and record the rationale in the guidance spec handed to engineering. The output is still a guidance spec per decision point — replacing ad-hoc "should we recommend or let them choose?" debates with a repeatable procedure — delivered as a standalone web canvas with facilitation guides and exportable spec sheets, plus Miro and Figma marketplace templates for distributed teams. The kit remains a design-time, human-facilitated methodology: not a runtime in-dashboard guidance layer, an adaptive assistant, or an automated design-feedback plugin.

Three evidence-driven additions to the spec sheets:

• Expert-targeting rule. In the proton therapy evaluation, gains in uncertainty awareness and new insights were strongest among longer-experience users. The user-mapping step now segments by expertise and budgets more detail-oriented guidance for expert segments instead of defaulting to simplification. Visualizationary corroborates expert receptivity: even seasoned visualization designers refined their work more effectively with natural-language guidance, suggesting well-framed guidance reads as help, not hand-holding — largely retiring the "experts will reject it" concern, though the evidence is still domain-limited.
• Honest success metrics. The same study found perceived confidence did not necessarily improve even when awareness and insights did. Spec sheets therefore prescribe measuring uncertainty awareness and new insights (task-based probes, insight counts) and explicitly warn teams not to treat a null confidence result as failure — or to promise confidence gains to stakeholders.
• Cognitive-load check. The climate-services study shows that visualizations stacking too many layers of complexity hinder decision-making by overloading attention, recognition, and working memory — and that redesigning uncertainty encoding plus well-chosen interactivity measurably reduced response time. A final worksheet pass now audits each specified comparison view and guidance layer for compounding complexity, and the kit adopts that study's measurement battery (response time, success ratios, eye-tracking, perceived effort) as its validation template before engineering build.

Why now

Paper [0] realizes its decision-based guidance method as practical worksheets, demonstrates it end-to-end on clustering, and relates it to existing guidance frameworks — a ready-to-adopt design procedure where there was only scattered research. The pain is documented: D-Tour and "Hey Dashboard!" both report that authoring guidance/onboarding is time-consuming, labor-intensive, and manually maintained. The new evidence upgrades the kit from a blank canvas to a validated pattern set: the proton therapy study supplies the first validated two-dimensional classification of uncertainty guidance, demonstrates co-design with domain experts inside a high-stakes regulated workflow, and shows that multi-alternative plan comparison — exactly what the kit's step (3) specifies — is a workable guidance pattern. The climate-services study supplies a quantified, user-centered evaluation methodology the kit can reuse wholesale.

What the evidence does not yet support

• Confidence is not a reliable success metric. The proton therapy study hypothesized guidance would raise perceived confidence; the observed gains were in awareness and insights, with confidence not necessarily moving. The kit adapts by making awareness/insight its default metrics and flagging confidence claims as unsupported.
• Generalization is plausible but unproven. The intrusiveness × detail-orientation taxonomy was validated in one domain (proton therapy planning). The climate study is adjacent — it validates user-centered redesign of uncertainty encoding for wind-energy climate services — but not the taxonomy itself. The kit therefore presents the two dimensions as a starting taxonomy and includes a lightweight per-domain pilot (using the measurement battery above) rather than claiming universal validity.
• Awareness is not outcomes. Whether raised uncertainty awareness correlates with better final decisions remains unmeasured; the kit's metrics are proximate and its spec sheets say so.
• Regulated-industry liability is unresolved. The proton therapy co-design process suggests expert co-design is the workable entry pattern for high-stakes domains, but the study evaluates perception and effectiveness, not liability; teams in regulated industries should still route specs through compliance review.
• Still open: whether worksheet-driven specs reduce engineering time-to-implementation versus ad-hoc design, and willingness to pay for a design-time methodology versus expecting it free inside Figma/Miro.