Product Talk · Free post · Discovery & customer research · Metrics, data & experimentation

Product in Practice: Getting Value Out of In-App Surveys Takes Iteration

Teresa TorresMay 15, 202416 min
SourceProduct Talk
KindFree post
PublishedMay 15, 2024
Originalproducttalk.org ↗
N:

Teresa Torres shares four examples from coach Ellen Juhlin of using in-app surveys to sustain customer touchpoints as part of continuous discovery. Across different products, Ellen tried a Mixpanel pop-up, an onboarding question, OneSignal recruiting flows, and Typeform rating prompts. Some efforts produced poor data, such as users dismissing ads-looking pop-ups or inaccurately self-describing their use case. The lessons focus on choosing tools that allow fast iteration, respecting user experience, avoiding speculative questions, and managing incentives. The piece matters because it shows continuous discovery can start small with low-cost tooling, even when results are imperfect.

01Key takeaways

  • Start continuous discovery with small in-app touchpoints rather than waiting for a perfect, large research program.
  • Prefer existing tools that let you change questions and targeting quickly without waiting on developer time.
  • Avoid asking speculative questions during onboarding; users cannot reliably predict how they will use a product yet.
  • Ask for personal details like email only after users have received value, to avoid stalling the survey.
  • Ask about behavior at the point it matters, such as for setup choices, rather than abstract intentions.
  • Do not reveal exactly what you are looking for in recruiting surveys, to prevent people gaming the screener.

02Key sections

Why small, recurring touchpoints matter
Continuous discovery means regularly integrating customer contact into weekly work rather than relying on intuition or stakeholder requests. The author argues teams can start small even when calendars are full.
Mixpanel pop-up for user types
A one-question pop-up asking work versus personal use was constrained by the tool's design and timing, and users mostly dismissed it. The team got too little usable data.
Onboarding question and its limits
Asking new users how they planned to use the app produced unreliable answers, because the question was speculative and users had gotten no value yet. Later recruiting revealed mismatches.
Recruiting with OneSignal, Typeform and Calendly
Targeted in-app prompts linked to a short Typeform survey filtered active users, then routed qualified people to a booking link. The setup was flexible but required coordination across several tools.
Typeform rating prompts for quality
A small tab with a 1-to-5 star rating and a follow-up question gathered quality feedback without blocking tasks, using a tool the team already owned. Consent was respected when reviewing user content.
Key learnings on tooling and experience
Ellen compares building custom tools, buying specialized tools, and using existing tools, favoring the last for speed of iteration. She also stresses timing, avoiding speculative questions, and not revealing what you seek in recruiting.

03From the post

“Committing to continuous discovery means changing the way your product team operates. It’s no longer about making decisions purely based on your intuitions or stakeholder requests, but finding ways to integrate touch points with customers into your work every week—if not every day. This can sound overwhelming. Your”

“asking users to self-describe their use case is not a reliable data point, especially when asked out of context.”Ellen Juhlin · Product Talk
“Just like in interviews, if you ask a speculative question, you will get an inaccurate answer that doesn't reflect reality”Ellen Juhlin · Product Talk
“You will be able to iterate faster with a tool that makes it easy to change the content and even targeting without requiring additional developer…”Ellen Juhlin · Product Talk

04Frameworks mentioned

Summary and takeaways written by PM Atlas; quotes are short excerpts. © the original author.