Q:How do I build AI products that people actually use?

Start by probing failure modes and defining quality bars before launch, then design for trust and product judgment. Marty Cagan and Marily Nika argue AI products are probabilistic, so the PM must set acceptable error rates and failure mitigations in the experience before shipping¹. Marily Nika adds that PMs should test with messy, ambiguous inputs to find where models break before users do⁷.

01Set quality bars early

• Define acceptable, delight, and do-not-ship quality levels, and estimate per-user cost at scale before trusting a demo⁷.
• Cagan and Nika say to match tolerance for error to the product: errors acceptable in a news feed are unacceptable in medication dosing¹.
• Design transparency so users understand what the AI can and cannot do and can trust it¹.

02Confirm AI is the right answer

• Teresa Torres advises confirming that AI is genuinely the right solution to the customer problem before building².
• Cagan warns against AI added only for marketing or competitive parity, and says value should be tested with both A/B tests and qualitative research¹.

03Build and observe the system

• Torres suggests treating AI feature work like orchestrating a team of interns: decompose tasks into smaller prompts and steps rather than one massive prompt².
• Log full traces of model interactions to observe and debug real behavior, and replace thumbs up/down with structured evals that score specific quality dimensions².
• Expect ongoing maintenance to be a significant burden².