Read the original at Lenny’s Newsletter ↗lennysnewsletter.com · subscriber post
By Lenny Rachitsky · lennysnewsletter.com · @lennysan on X · YouTube · LinkedIn
Marily Nika, a former AI PM at Google and Meta, presents a weekly ritual for building AI product sense: probing a model with messy, ambiguous, and difficult inputs to map failure modes before users do. She argues PMs should define a minimum viable quality bar that includes cost, and design guardrails so failures are predictable and recoverable.
Subscriber post — summary only01Key takeaways
- Test AI features with messy, obviously wrong inputs to see where models confidently invent structure.
- Compare a model's flawed output with an output given explicit expected behavior to surface product requirements.
- Probe ambiguous prompts to find where models misread intent and where the product should ask clarifying questions.
- Define acceptable, delight, and do-not-ship quality bars early, adjusted to strategic context such as risk and phase.
- Estimate the per-user cost envelope at scale before falling for a demo, and design guardrails like asking for clarification or admitting uncertainty.
02Frameworks mentioned
Summary and takeaways written by PM Atlas; quotes are short excerpts. © the original author.