Anthropic’s first technical PM on token maxing, the jagged edge, and living in the future

Diane Penn, head of product for AI research and Labs at Anthropic and its first technical PM, discusses the company's early days, the shift from PRDs toward evals as the core product artifact, and how research and product teams work together. The conversation also covers the Labs team's bet-making approach, the growing importance of judgment and user-centric work, and personal practices for staying productive and sane amid rapid AI progress.

01Key learnings

• Treat evals as the new PRDs: turn vague user complaints into concrete, reproducible test sets that researchers can act on and measure against across model versions.
• Dig into user transcripts, not just pixels, to find where the model actually failed, such as wrong tool calls, bad retrieval, or overconfidence, before deciding what to fix.
• Keep strong conviction about a theme or area while staying loose about the specific prototype, and revisit bets across one to two model generations as capabilities shift.
• Stay hands-on with the technology; managers should personally ship and own a work stream so they keep an accurate sense of how fast models are improving.
• Ask how the product would change if a much more capable model arrived, and build for forward compatibility rather than today's limitations.
• Experiment communally: working in public and pairing with curious colleagues surfaces use cases faster than solo exploration and keeps experimentation enjoyable.
• Use AI as a thinking partner that pushes back: form your own point of view first, then let the model challenge and sharpen it rather than simply agreeing.
• Judgment, persistence, curiosity, and user-centric depth are the human traits that stay valuable as AI takes over more execution work.

02Frameworks mentioned

Summary and takeaways written by PM Atlas; quotes are short excerpts. © the original author.