"Hey Dashboard!": Supporting Voice, Text, and Pointing Modalities in Dashboard Onboarding using Large Language Models
Vaishali Dhanoa, Gabriela Molina León, Eve Hoggan, Eduard Gröller, Marc Streit, Niklas Elmqvist · 2026 · 1 citationsRead the paper
Visualization dashboards are regularly used for data exploration and analysis, but their complex interactions and interlinked views often require time-consuming onboarding sessions from dashboard authors. Preparing these onboarding materials is labor-intensive and requires manual updates when dashboards change. Recent advances in multimodal interaction powered by large language models (LLMs) provide ways to support self-guided onboarding. We present Diana (Dashboard Interactive Assistant for Navigation and Analysis), a multimodal dashboard assistant that helps users for navigation and guided analysis through chat, audio, and mouse-based interactions. Users can choose any interaction modality or a combination of them to onboard themselves on the dashboard. Each modality highlights relevant dashboard features to support user orientation. Unlike typical LLM systems that rely solely on text-based chat, Diana combines multiple modalities to provide explanations directly in the dashboard interface. We conducted a comparative qualitative user study to understand the use of different modalities for different types of onboarding tasks and their complexities.
What the full text shows
Working prototype- Participants using Diana achieved higher task accuracy and completeness compared to those using a static dashboard guide. (Comparative user study with 12 participants (6 with Diana, 6 with static guide))
- Users frequently preferred speech-based interaction over keyboard or mouse for onboarding help and follow-up questions. (Main study participants using Diana)
- The radial menu was the least used feature, described as unintuitive and time-consuming. (Main study participants using Diana)
- The system successfully detected non-native English speakers and switched to their understood language. (Formative study using OpenAI Realtime API)
- Setup
- 12 participants (ranging from novice to expert) tested on a single Microsoft Power BI dashboard.
- How
- The researchers developed Diana, a multimodal assistant using LLMs and REST APIs to provide chat, audio, and visual highlights on dashboards. They conducted a two-phase study consisting of a formative study (n=4) and a main qualitative user study (n=12) comparing Diana against a static printed guide.
- Limitations
- Low usage and unintuitive design of the radial menu. · Occasional lack of synchronization between visual highlights and voice output. · High usage-based costs associated with the OpenAI Realtime API. · Findings are limited to a single dashboard and the Power BI platform.
Summarized by Gemma 4 from the open-access full text · 9 Oct 2026. Read it