By Teresa Torres · producttalk.org · @ttorres on X · LinkedIn
Teresa Torres argues that generative AI changes many product-management practices (prompt engineering, context engineering, orchestration, and evals) but that core fundamentals still determine whether a product succeeds. She explains LLMs as next-token predictors that are powerful yet confidently wrong, then walks through the new skills needed to build reliable AI features. She concludes that these skills only matter if teams first solve the right customer problem through discovery, and that iterative prototyping and ethical data practices remain essential. The talk matters because it separates genuinely new AI craft from timeless product discipline.
01Key takeaways
- Learn how LLMs predict outputs so you can anticipate where they will fail confidently.
- Design one-shot prompts with a role, explicit output format, and detailed instructions for reliable production behavior.
- Supply only the context a model needs for each task, using retrieval rather than dumping all available data.
- Break complex AI tasks into orchestrated steps, since simpler focused calls produce higher-quality results.
- Start evals small with 20-100 real examples, code assertions, LLM-as-judge, and human review, then add sophistication.
- Do discovery before building: AI speed makes skipping the question of what to build more tempting and more costly.
02Key sections
- How LLMs work and why they fail
- LLMs predict the next token from learned patterns, which makes them powerful but prone to confident mistakes. Understanding this mechanism explains why their outputs can be both impressive and unreliable.
- New AI skills for building products
- Prompt engineering, context engineering, orchestration, and evals are the new technical skills needed when AI is a feature rather than a chat tool. Each shifts the PM role toward specifying inputs, workflows, and quality measurement.
- Discovery still comes first
- AI makes building fast, which tempts teams to skip asking what should be built. Clear outcomes, customer understanding, and assumption testing remain the foundation.
- Prototype, test, and iterate
- Torres's Interview Coach went through several rebuilds as she tested assumptions about feasibility, interface, and reliability. Shipping small experiments de-risks AI features better than building everything upfront.
- Ethical data practices
- Tracing AI interactions for debugging can expose sensitive user data, so products should require explicit consent and build privacy in from the start.
03From the post
“Every product team is wrestling with the rapid change brought on by generative AI. It's impacting our roadmaps and how we do our jobs. But not everything is changing. Many of the fundamentals stay exactly the same. Last week, I delivered this talk as part of The Future of”
04Frameworks mentioned
Summary and takeaways written by PM Atlas; quotes are short excerpts. © the original author.