Product in Practice: Iterating on Outcomes with Limited Data

Teresa Torres profiles Thomas Groendal, a senior PM at Bluestone Analytics, whose intelligence-analysis products (DarkBlue and DarkPursuit) operate on the dark web with few users, privacy restrictions, and very long sales cycles. Because customer data was limited, Thomas started with blunt metrics and anecdotal feedback, then instrumented the product to test candidate outcomes like clue captures and copy events. A retention analysis showed that repeated copying predicted return visits far better than clue captures, so the team reframed copy events as the outcome and clue captures as a health metric. The story matters because it shows how to choose a meaningful outcome when data is scarce and intuition about user value can be wrong.

01Key takeaways

• In low-data environments, start with any measurable signal and iterate repeatedly rather than waiting for perfect information.
• When a traction metric strongly signals customer value, it can serve as the outcome, but pair it with a counterweight metric to prevent gaming.
• Test candidate outcomes against retention to see which behaviors actually predict return use, rather than relying on team intuition.
• Long sales cycles can be bridged by using retention as a leading proxy for product-market fit.
• Don't assume users treat your product as the center of their workflow; some value comes from narrow, quick tasks such as confirming a null search result.

02Key sections

The Challenge of Defining an Outcome

Outcomes must be influenceable by the product team and tied to business results, which is hard when user counts are small and tracking is restricted. Thomas's product faced both constraints.

Zero Information Phase

The team began with a weak satisfaction rating and blunt login measures, relying mostly on anecdotal and qualitative evidence until they could instrument the product.

Testing Candidate Outcomes

Clue captures matched the team's intuition about ideal usage, but they were slow to measure, so copy events were added as a faster signal. Teresa advised pairing copy events with a counterweight metric to avoid gaming.

Connecting Behavior to Retention

Because purchases took up to 18 months, the team used retention as a proxy and found that copying five or more times strongly predicted return visits, overturning their original assumption.

Key Takeaways

Thomas emphasizes persistence through low-information iteration and humility about assuming the product's central value to users.

03From the post

“We often hear from people that building an opportunity solution tree is hard work. And perhaps one of the most challenging parts of building opportunity solution trees is defining the outcome that you put at the top. As a quick reminder, your outcome should be something the product team is”

04Frameworks mentioned

Summary and takeaways written by PM Atlas; quotes are short excerpts. © the original author.