Drawer 05 · 298 notes · 90 people · 2014–2026

Building AI products

AI-native products, evals, agents, and how AI changes product work

Q:
Answers from Building AI products notes, cited
Lenny’s Podcast73 min

How 80,000 companies build with AI: products as organisms, the death of org charts, and why agents will outnumber employees by 2026

Asha Sharma · Aug 28, 2025
Key learnings
  • Treat AI products as living systems: build the loop of collecting usage signals, defining rewards, A/B testing, and fine-tuning…
  • Invest in post-training and fine-tuning on your own data, not just off-the-shelf models; once models are large, adapting them to…
  • Watch for a shift from GUIs toward composable, code-native interfaces, since text streams and composability connect better with…
Lenny’s Podcast100 min

How Block is becoming the most AI-native enterprise in the world

Dhanji R. Prasanna · Oct 26, 2025
Key learnings
  • Track AI impact with self-reported time savings plus validation metrics such as PR throughput and feature delivery, which Block…
  • Expect the current AI value baseline to keep rising; adopt tools continuously and re-evaluate where they add value as…
  • Non-technical staff building their own small internal tools with agents can compress weeks of waiting on engineering queues into…
Lenny’s Podcast118 min

The AI paradox: More automation, more humans, more work

Dan Shipper · May 24, 2026
Key learnings
  • Keep using the newest models and agent tools for your real work, and experiment on your own time if your employer restricts…
  • Expect most company AI to start as one shared agent that everyone can talk to, likely in Slack, rather than many personal agents…
  • Design products so agents can be users too: make them easy to drive via CLI and browser-friendly, because the customer's own AI…
Lenny’s Podcast107 min

Building a magical AI code editor used by over 1 million developers in four months: The untold story of Windsurf

Varun Mohan · Apr 20, 2025
Key learnings
  • Expect your first big bet to be wrong; stay irrationally optimistic about the vision while ruthlessly testing and discarding the…
  • Be willing to pivot fully and quickly when hypotheses break, as Varun's team did when model architectures converged on…
  • Build the interface where AI-generated code gets reviewed, not just where it gets typed; Windsurf's tripled acceptance rate after…
Lenny’s Podcast71 min

Microsoft CPO: If you aren’t prototyping with AI, you’re doing it wrong

Aparna Chennapragada · May 18, 2025
Key learnings
  • Prototype and build to see what you want before committing; prompt sets and prototypes can replace traditional PRDs as the…
  • Time to first demo is shrinking while time to full deployment grows, so raise the bar for what earns scale and avoid chasing…
  • Treat conversational interfaces as designed products: prompts, editable plans, progress display, and follow-up suggestions are…
Lenny’s Podcast115 min

Why half of product managers are in trouble

Nikhyl Singhal · Apr 19, 2026
Key learnings
  • Expect the information-moving part of product work to be automated; judgment about what to build and whether changes are good…
  • Build with AI tools to solve your own daily problems; you don't need to be an engineer, you need to know what good looks like.
  • Aim to obsolete yourself: automate the parts of your job you dislike, as the best engineers do.
Lenny’s Podcast115 min

Max Schoenig

Max Schoenig
Key learnings
  • Prototyping in code forces you to understand the medium, so designers and PMs gain better judgment about what the real product…
  • Agency, the willingness to treat your company and role as malleable, matters more than any single skill as AI tools make…
  • Start by making things; tinkering builds the belief that the world around you can be changed, which Max sees as the root of…
Lenny’s Podcast121 min

How we restructured Airtable’s entire org for AI

Howie Liu · Aug 31, 2025
Key learnings
  • Leaders should stay close to the product details, since in AI-era products the interaction design and underlying behavior are the…
  • Use AI products constantly, even multiple times an hour, and build small weekend projects to understand what models can do and…
  • Ask whether you would build the same mission from scratch as an AI-native company; if your existing assets don't give a real…
Lenny’s Podcast110 min

Aishwarya Naresh Reganti + Kiriti Badam

Aishwarya Naresh Reganti + Kiriti Badam · Jan 11, 2026
Key learnings
  • AI products differ from traditional software mainly because user inputs and model outputs are both non-deterministic, so behavior…
  • Each time you grant an AI system more decision-making autonomy, you give up some human control; autonomy should be earned through…
  • Start with low-agency, high-human-control versions (e.g., suggestions to human agents) and log human corrections to build a…
Lenny’s Podcast68 min

Behind the product: Replit

Amjad Masad · Nov 21, 2024
Key learnings
  • Replit aims to remove the setup friction of building software by bundling the editor, runtime, packages, database, and deployment…
  • Once AI removes the build bottleneck, the limiting factor shifts to how fast you can generate and articulate good ideas, so…
  • Product managers can use an AI agent to build a v1 prototype, test it with real users, and then hand a validated concept to…
Lenny’s Podcast153 min

From ChatGPT to Instagram to Uber: The quiet architect behind the world’s most popular products

Peter Deng · Jun 22, 2025
Key learnings
  • Before scaling past product-market fit, invest in systems and architecture that let the team move sustainably faster, since…
  • Build a growth team early; a growth leader forces instrumentation and rigorous questioning of what is actually happening in the…
  • Balance growth-focused and craft-focused people deliberately, so the tension between metrics and product quality produces…
Lenny’s Podcast103 min

Gustav Söderström

Gustav Söderström · May 21, 2023
Key learnings
  • Expect the internet's next shift from recommendation to generation to require rethinking products and business models, much as…
  • Design UIs to match your model's real accuracy: if predictions are right only one in four or five times, show several options at…
  • Treat generative AI as a distinct paradigm rather than more of the same machine learning, and ask what experience couldn't exist…
Lenny’s Podcast98 min

Everyone’s an engineer now: Inside v0’s mission to create a hundred million builders

Guillermo Rauch · Apr 13, 2025
Key learnings
  • Treat taste as a skill you can build by increasing exposure hours: deliberately spend time watching people use your product and…
  • Learn how systems work under the hood, since knowing the right technical tokens helps you steer AI models toward your intent more…
  • Put product feedback loops inside the product itself, as Rauch suggests borrowing from Stripe's in-product feedback button…
Lenny’s Podcast91 min

How Intercom rose from the ashes by betting everything on AI

Eoghan McCabe · Aug 21, 2025
Key learnings
  • When growth stalls in a late-stage SaaS business, cut costs hard and pick one clear strategic lane instead of trying to serve…
  • Simplify and fairly price your product even at short-term revenue cost; McCabe gave away roughly $50M ARR to replace confusing…
  • Price AI products on outcomes that align with customer value, like charging 99 cents per resolved ticket, rather than on the…
Lenny’s Podcast67 min

The future of AI in software development

Inbal S · Dec 1, 2023
Key learnings
  • Engineers using AI tools need to shift from writing code to systems and architecture thinking, which lets junior developers spend…
  • Don't adopt AI for its own sake; start from the customer problem and ask how AI can best solve it, rather than asking what to do…
  • AI tools must fade into the background: any extra setup, asking, or waiting adds friction and developers will abandon the tool.
Lenny’s Podcast110 min

Why great AI products are all about the data

Shaun Clowes · Dec 29, 2024
Key learnings
  • Start every document from the customer, market, and competitor perspective rather than internal execution, since…
  • Right-size qualitative research: interview roughly 7 to 14 people, because fewer gives too little signal and more stops yielding…
  • Use LLMs to hunt for where your strategy doesn't fit customer feedback, not to confirm what you already believe, and to infer…
Lenny’s Podcast105 min

AI’s third era: the rise of persistent AI coworkers

Tara Seshan · Aug 30, 2026
Key learnings
  • Be prolific and empirical rather than theoretical: get to something testable with users as fast as possible instead of writing…
  • Sharpen the single most important hypothesis (the 'eigenquestion') and design fast tests to resolve it; that loop has always been…
  • Build for where models will be in two to three months: building for current capabilities or for distant future capabilities are…
Lenny’s Podcast100 min

What world-class GTM looks like in 2026

Jeanne Grosser · Nov 30, 2025
Key learnings
  • Treat go-to-market as an integrated lifecycle spanning marketing, sales, customer success, and support, mapping the jobs to be…
  • Expect the buyer experience of being sold to to become a differentiator, so design the sales journey as a distinctive, valuable…
  • Frame value around avoiding pain and reducing risk, since many buyers purchase to avoid downside rather than chase upside…
Lenny’s Podcast81 min

Why experts writing AI evals is creating the fastest-growing companies in history

Brendan Foody · Sep 18, 2025
Key learnings
  • Treat evals as the product requirement document for a model: if you cannot measure what success looks like for a task, you cannot…
  • Companies should build a systematic test of how AI automates their core value chain, since that measurement is the prerequisite…
  • Evals also serve as sales collateral, showing customers and researchers concretely which real-world capabilities a model or…
Lenny’s Podcast105 min

Sherwin Wu V2

Sherwin Wu V2 · Feb 12, 2026
Key learnings
  • Build for where models are going rather than where they are today, since products that are almost-working now can become…
  • Treat fast-changing AI scaffolding skeptically: models often absorb tooling like vector stores and agent frameworks, so avoid…
  • Don't blindly follow customer feature requests in AI; customers may anchor on local maxima while the underlying models are…
Lenny’s Podcast54 min

TiboSottiaux

TiboSottiaux
Key learnings
  • Expect most actions on the internet to be taken by agents, so build products that can serve agent traffic at scale and handle the…
  • Design for the capabilities you expect in roughly a year (about 10x better than today), not just what current models can do, to…
  • Favor systems that learn from your goals and feedback over hand-tuned agent loops and workflows, which Tibo thinks will be…
Lenny’s Podcast142 min

An AI state of the union: We’ve passed the inflection point, dark factories are coming, and automation timelines

Simon Willison · Apr 2, 2026
Key learnings
  • Coding agents crossed a reliability threshold around November 2025, so they now usually do what you ask, which changed how…
  • Writing code is now cheap, so the bottlenecks move to ideation, prototyping, testing, and process; build several quick prototypes…
  • Use test-driven development with agents: have them write tests, run them, and watch them fail first (Red/Green TDD) to keep…
Lenny’s Podcast93 min

Lessons in product leadership and AI strategy from Glean, Google, Amazon, and Slack

Tamar Yehoshua · Sep 26, 2024
Key learnings
  • Excel in your current role before chasing the next one; advancement follows demonstrated impact, not just hitting goals or…
  • Evaluate a potential employer's engineering partner before joining, since great ideas that can't be built lead nowhere.
  • Don't assume a company must be well run to succeed; hyper-growth firms often run chaotically, but prioritize companies whose…
Lenny’s Podcast102 min

How we built Grok Bot in a month

Roman Ugarte · Sep 8, 2026
Key learnings
  • Build a new product from scratch with a small, isolated team rather than retrofitting an existing surface, so each micro-decision…
  • Onboard early users by hand and sit in on calls; fixing failures the same day compresses learning dramatically.
  • Let users reveal patterns on their own before productizing them, then gently encourage the winning patterns without making them…