We were confidently wrong
While refining the agent desktop at Questrade, we had two strong hypotheses about what slowed agents down: screen toggling between systems, and poor case tracking (no proper case IDs, which made escalations painful). Weeks of one-on-one interviews with agents confirmed both. So we started moving toward solutions. Sensible, right?
Both hypotheses were about to lose to things nobody had mentioned in a single interview.
Changing the question
Instead of asking agents about their pain again, I ran a structured workshop. We walked them through a range of modern agent desktops, from inside and outside the industry, and just watched what they reacted to. Not "describe your problems" but "here are alternatives, what makes you lean forward?"
What lit them up was nothing we'd planned for: auto-summarization of calls, and fluid switching between multiple active cases. The case-ID issue only mattered during rare escalations. The daily pain was being stuck in a linear, single-thread workflow with no memory. Agents had never articulated it, because they'd never seen the alternative.
What changed
We reprioritized the roadmap. Multi-case workflow support moved up, and auto-summarization went into active exploration. More durably, the workshop format itself became a habit: internal agents treated as a critical user group with structured comparative feedback loops, not just ad hoc interviews.
One workshop produced more signal than weeks of interviews. Not because the interviews were bad, but because comparison unlocks reactions that description can't.
What this shows
Research methodology as product judgment: knowing when your validated hypotheses still deserve suspicion, and creating the space where users can show you what they can't tell you.