Product in Practice: Testing Assumptions Was Tricky But the Convo Team Didn’t Give Up

Convo's product team set out to test assumptions about a feature letting Deaf users bring an interpreter into a Zoom meeting, but running those tests over Zoom introduced confusing meta-tasks and technical problems. Rather than abandon assumption testing, they moved to in-person sessions: a colleague first tested at a Deaf community event, then Amanda recruited 20 Deaf students and staff at Gallaudet University in five hours. Removing the Zoom confusion produced cleaner data, and the test revealed that their core solution assumption failed, saving them from building a flawed experience. The story shows how persistence, flexible recruiting, and careful test design can protect a team from costly mistakes.

01Key takeaways

• Test upstream assumptions first so you don't waste effort on later steps that depend on disproven earlier ones.
• Remove confusing test setups, such as asking participants to use the same tool inside the tool being tested.
• Recruit where your target users already gather, like schools, events, or community organizations, to get volume quickly.
• Prepare mock materials and written test steps in advance to keep sessions consistent across facilitators.
• Use a prototyping platform's session summaries alongside observer notes to build a full picture of each test.
• Catching a failed assumption before building can save significant engineering time and prevent user frustration.

02Key sections

Context and the Zoom problem

The team, serving Deaf users through an interpreting service, had to test a flow involving both their app and Zoom. Asking participants to interact with a prototype while already inside Zoom created confusion that obscured the assumptions being tested.

Early in-person experiment

Jason tested at a Deaf community event where participants scanned a QR code to open a prototype on their own phones. Familiar faces and built-in downtime let him reach 18 users without incentives.

Recruiting at Gallaudet University

Amanda set up a booth in a high-traffic student area, proactively approaching people and offering snacks for brief participation. She completed 20 tests in about five hours, and on-the-spot recruiting avoided no-shows.

Results and logistics

Testing in person removed the meta confusion and allowed A/B testing of button icons. The key assumption failed, forcing a redesign, and the total cost and lead time were modest.

Key learnings

The team highlights testing upstream assumptions first, building efficient recruiting pipelines, and the value of catching flaws before engineering investment.

03From the post

“Identifying and testing assumptions is a critical part of continuous discovery. But what happens when your assumption tests don’t go as planned? Whether you encounter technical difficulties, have a hard time finding customers to connect with, or run up against any other number of problems, it can be tempting”

04Frameworks mentioned

Summary and takeaways written by PM Atlas; quotes are short excerpts. © the original author.