Prototype Testing

Marty Cagan argues that testing high-fidelity prototypes with real users is the most important activity a product manager performs, because it reveals whether people can use a product and whether they want it before engineering invests months building it. He explains that even without a dedicated usability team or budget, PMs can run effective tests themselves. The article covers recruiting participants through many low-cost channels, preparing tasks and value-focused questions, setting up a test environment that keeps users natural, and running sessions where the facilitator stays quiet and observes. It closes with guidance on iterating quickly on feedback and knowing when enough testing is done.

01Key takeaways

• Test prototypes with real users yourself, even with minimal budget, since this is among the highest-value activities a PM can do.
• Capture users' existing mental models and current solutions before showing the prototype, because that first-time perspective disappears quickly.
• Measure desirability as well as usability by asking how likely users are to recommend the product and how much they'd pay, tracked on a scale over time.
• Attend every session yourself and keep quiet, parroting user actions instead of leading them or asking for design critiques.
• Iterate on problems as soon as you spot them, and treat it as a win when a weak idea is shelved before engineering builds it.

02Key sections

Why prototype testing matters

Prototypes are valued mainly as a way to deepen understanding and validate ideas before building, with the secondary benefit being a clearer spec for the team. Testing covers both usability and desirability.

Finding test subjects

Lists practical recruiting channels such as charter customers, trade shows, online ads, friends and family, customer lists, and recurring in-office test sessions. It also notes compensation norms and how personal phone reminders reduce no-shows.

Preparing the test

Defines key tasks to test and stresses capturing how users approach the problem before seeing the prototype. Interview questions should gauge how much users value the product, including recommendation likelihood and willingness to pay on a scale.

Test environment and roles

Testing can happen in a labs, a coffee shop, or a customer's office, with face-to-face preferred over remote tools. The PM should attend every session, ideally with a separate note-taker, since summaries miss key insights.

Running the sessions

Keep users in use mode rather than critique mode, stay silent through struggle, and parrot back what users do to avoid leading them. Observe body language and note whether users completed tasks easily, struggled, or gave up.

Iterating on the prototype

Fix problems as soon as they appear rather than waiting for a full round of testing, and consider stopping when an idea cannot find traction. Credits Steve Krug and Creative Good for shaping the approach.

03From the post

“A partnership dedicated to teaching best practices to product teams and product leaders”

04Frameworks mentioned

Summary and takeaways written by PM Atlas; quotes are short excerpts. © the original author.