Drawer 05 · 298 notes · 90 people · 2014–2026

Building AI products

AI-native products, evals, agents, and how AI changes product work

Q:
Answers from Building AI products notes, cited
Lenny’s Podcast116 min

OpenAI’s CPO on how AI changes must-have skills, moats, coding, startup playbooks, more

Kevin Weil · Apr 10, 2025
Key learnings
  • Build products at the edge of current model capabilities, since models improve every few months and a barely-working product can…
  • Treat evals as unit tests for models: define hero use cases, write evals for them, and hill-climb on those evals to know whether…
  • Expect AI behavior to be fuzzy: a model right 60%, 95%, or 99.5% of the time implies very different product designs, so study…
Lenny’s Podcast128 min

Inside ChatGPT: The fastest-growing product in history

Nick Turley · Aug 9, 2025
Key learnings
  • Ship early to learn what people actually want; with AI, the valuable behaviors are emergent, so you won't know what to polish…
  • Treat the model itself as a product: iterate on it using real user use cases, personality/vibe checks, and new capabilities like…
  • Set the team's pace deliberately by asking whether work is 'maximally accelerated', separating fast product iteration from…
Lenny’s Podcast93 min

Anthropic’s CPO on what comes next

Mike Krieger · Jun 5, 2025
Key learnings
  • Use AI as an independent strategy critic: ask it to be brutal or to challenge your thinking, since default prompts tend to…
  • As AI writes most code, bottlenecks shift upstream to deciding what to build and aligning people, and downstream to merge queues…
  • Give teams a minimum viable strategy so people feel empowered to build and explore at the edge of model capabilities without…
Lenny’s Podcast153 min

Marc Andreessen: The real AI boom hasn’t even started yet

Marc Andreessen · Jan 29, 2026
Key learnings
  • Expect AI to make already-skilled people far more capable; the gains are largest for those who pair deep expertise with AI tools.
  • Rather than fearing job loss, focus on task loss: jobs persist while the individual tasks composing them shift, which is how…
  • Combine at least two or three domains deeply (e.g., coding, product, design); the combination of skills creates far more value…
Lenny’s Podcast128 min

The AI-native startup: 5 products, 7-figure revenue, 100% AI-written code

Dan Shipper · Jul 17, 2025
Key learnings
  • Non-programmers can use Claude Code-style command-line agents with local files to process large text sets, like meeting notes or…
  • Hire or assign someone to an AI operations role who continuously turns repetitive team tasks into prompts and workflows, and make…
  • Apply 'compounding engineering': spend a bit of effort turning each recurring task (like writing PRDs) into a reusable prompt or…
Lenny’s Podcast109 min

A rational conversation on where AI is actually going

Benedict Evans · May 31, 2026
Key learnings
  • Treat AI as roughly as big as the internet or mobile, and recognize we are early, like 1997, with most applications not yet built…
  • Expect adoption to be very uneven; survey data shows many people are not using AI at all, so don't assume the people in your…
  • Ask whether a job is a task that can be automated or a deeper outcome; consultants and professionals are hired for judgment…
Lenny’s Podcast89 min

The rise of Cursor: The $300M ARR AI tool that engineers can’t stop using

Michael Truell · May 1, 2025
Key learnings
  • Build the product you and your team actually use every day (dogfooding) so you never ship anything that isn't genuinely useful…
  • When working with AI coding tools, chop tasks into small specify-generate-review loops rather than handing over one giant…
  • Spend time on side projects deliberately pushing AI to its limits to build a calibrated gut feeling for what current models can…
Lenny’s Podcast111 min

He saved OpenAI, invented the “Like” button, and built Google Maps: Bret Taylor on the future of careers, coding, agents, and more

Bret Taylor · Jul 31, 2025
Key learnings
  • Don't just digitize what existed before; rebuild the experience natively for the new platform so customers have a reason to…
  • Ask yourself what the most impactful thing is today, and reflect on whether you're choosing it out of comfort or because it's…
  • Be skeptical of your own storytelling: customer reasons given to salespeople can mask deeper product problems, so seek…
Lenny’s Podcast64 min

AI and product management

Marily Nika · Feb 5, 2023
Key learnings
  • Avoid the shiny object trap: only add AI when there is a real, well-defined pain point worth solving, and identify the problem…
  • Don't use AI for an MVP; fake the AI behavior with a clickable prototype and test desirability with users first.
  • Use ChatGPT as a sounding board for sharpening mission statements and generating user segments, personas, and motivations, while…
Lenny’s Podcast79 min

Building Lovable: $10M ARR in 60 days with 15 people

Anton Osika · Mar 9, 2025
Key learnings
  • Be precise and explicit when prompting an AI builder: say exactly what you expect and which parts are not working, rather than…
  • Use the agent's chat mode to ask how something works or why it isn't doing what you want; this builds understanding while you…
  • Expect to spend a full week taking one real problem from idea to a working product that people use; Anton suggests that alone…
Lenny’s Podcast100 min

Jenny Wen

Jenny Wen · Mar 1, 2026
Key learnings
  • Classic linear design process (research, diverge, converge) is breaking down as engineers ship prototypes quickly; designers…
  • Design work is splitting into two modes: supporting implementation and setting a vision, where the vision now often spans three…
  • Designers can use coding tools to do last-mile polish and prototype in real code, rather than waiting on engineers for every…
Lenny’s Podcast117 min

AI prompt engineering in 2025: What works and what doesn’t

Sander Schulhoff · Jun 19, 2025
Key learnings
  • Prompt engineering remains valuable: Sander argues that bad prompts can drop accuracy near zero on a task while good prompts can…
  • Few-shot prompting is the highest-impact basic technique: give the model several examples of what good output looks like, using a…
  • Ask the model to decompose a problem by first listing the subproblems it needs to solve, then solving each one before answering…
Lenny’s Podcast134 min

From skeptic to true believer: How OpenClaw changed my life

Claire Vo · Mar 29, 2026
Key learnings
  • Run several purpose-built agents instead of one general agent; splitting tasks keeps each agent's context focused and reduces…
  • Install OpenClaw on a separate clean machine such as an old laptop or Mac Mini, with its own local admin account, rather than…
  • Onboard the agent like a new employee: give it its own email and calendar, share access selectively, and expand trust…
Lenny’s Podcast131 min

Why AI evals are the hottest new skill for product builders

Hamel Husain & Shreya Shankar · Sep 25, 2025
Key learnings
  • Evals are systematic ways to measure and improve an AI application, essentially data analytics on LLM behavior, replacing…
  • Start with error analysis by manually reviewing around 100 sampled traces and writing short notes on the first upstream failure…
  • Product people with domain expertise should lead open coding; appoint one 'benevolent dictator' whose judgment you trust instead…
Lenny’s Podcast87 min

Netflix CPTO on AI and the future of product and tech roles

Elizabeth Stone · Jul 19, 2026
Key learnings
  • Expect a 'storming' phase with new technology; don't abandon AI, but be deliberate about capturing benefits while limiting costs…
  • Let product, design and data people prototype early against a clearly defined business problem, while still working with…
  • Invest in source-of-truth data, guardrails for shipping and testing, and review processes where AI output needs verification, and…
Lenny’s Podcast100 min

The non-technical PM’s guide to building with Cursor

Zevi Arnovitz · Jan 18, 2026
Key learnings
  • Start slowly with a ChatGPT project acting as a skeptical 'CTO' before moving to coding tools, gradually easing into code to…
  • Capture ideas quickly as Linear issues using reusable slash commands, so you can pick them up later with context already gathered.
  • Separate the workflow into explore, plan, execute, review, and document phases, and have the AI ask clarifying questions before…
Lenny’s Podcast108 min

How Anthropic’s product team moves faster than anyone else

Cat Wu · Apr 23, 2026
Key learnings
  • Shorten the path from idea to user: AI-era timelines have shrunk from six months to days or a week, so PMs should focus on…
  • Ship features as clearly labeled research previews to lower the commitment of launching and get fast user feedback that can be…
  • Set a clear, specific goal for who the key user is and what success means, which rules out many approaches and speeds decisions.
Lenny’s Podcast78 min

The Godmother of AI on jobs, robots & why world models are next

Dr. Fei Fei Li · Nov 16, 2025
Key learnings
  • Large labeled datasets were the missing ingredient that unlocked modern AI; big data, neural networks, and GPUs together formed…
  • Pursue a north-star problem and commit to it for years, as ImageNet's object recognition focus did for her lab.
  • Be willing to take intellectual and career risks, such as leaving tenure track or joining new ventures, when the mission and…
Lenny’s Podcast97 min

The 100-person AI lab that became Anthropic and Google's secret weapon

Edwin Chen · Dec 7, 2025
Key learnings
  • Define quality in rich, specific terms for each domain (e.g., what makes a poem Nobel-worthy, not just whether it has eight…
  • Gather thousands of signals on each worker and task, from background and expertise to performance, and use them like an ML…
  • Don't trust benchmarks at face value: many contain wrong answers, and optimizing for them can make models worse at messy…
Lenny’s Podcast102 min

Al Engineering 101 with Chip Huyen (Nvidia, Stanford, Netflix)

Chip Huyen · Oct 23, 2025
Key learnings
  • Focus on what actually improves AI apps, such as talking to users, improving data, writing better prompts, and optimizing…
  • Be cautious about committing to newly released technologies that have not been widely tested, since switching away from them…
  • Fine-tuning and post-training can shape model behavior a lot, and many teams now focus effort there because base pre-training…
Lenny’s Podcast123 min

The rise of the professional vibe coder (a new AI-era job)

Lazar Jovanovic · Feb 8, 2026
Key learnings
  • Start a project with several parallel attempts (brain dump, typed prompt, design reference, code template) to find the strongest…
  • Spend most of your time planning and chatting with the AI rather than executing, since clarity matters more than raw build speed.
  • Write a set of source-of-truth docs (master plan, implementation plan, design guidelines, user journeys, tasks.md) so the agent…
Lenny’s Podcast102 min

Anthropic’s first technical PM on token maxing, the jagged edge, and living in the future

Dianne Penn · Jul 26, 2026
Key learnings
  • Treat evals as the new PRDs: turn vague user complaints into concrete, reproducible test sets that researchers can act on and…
  • Dig into user transcripts, not just pixels, to find where the model actually failed, such as wrong tool calls, bad retrieval, or…
  • Keep strong conviction about a theme or area while staying loose about the specific prototype, and revisit bets across one to two…
Lenny’s Podcast102 min

What AI means for your product strategy

Paul Adams · Oct 26, 2023
Key learnings
  • Before redesigning a product for AI, start from the core premise: why people use it and what problem it solves, then map what AI…
  • Treat AI as a strategic shift rather than a feature; Intercom ripped up its strategy and rebuilt from first principles after the…
  • Build real machine learning depth on the team, but spread AI skills across product teams instead of isolating AI work in a…
Lenny’s Podcast148 min

Anthropic’s $1B to $19B growth run: how Claude became the fastest-growing AI product in history

Amol Avasare · Apr 5, 2026
Key learnings
  • Expect most growth time to go to 'success disasters' once growth is rapid; firefighting breaking systems matters as much as new…
  • In AI-first products, skew the growth portfolio toward larger swings, since future product value could be orders of magnitude…
  • Treat activation as a top lever in AI products: identify user traits that predict which feature fits them, then guide them there.