Drawer 05 · 298 notes · 90 people · 2014–2026

Building AI products

AI-native products, evals, agents, and how AI changes product work

Q:
Answers from Building AI products notes, cited
@ttorres · Teresa Torres on X♥ 97
"The only way to know if our AI products and workflows are any good is with evals." 💡 If you're using AI to write PRDs, analyze customer feedback, or build customer-facing AI products, you need to understand evals. This guide breaks down what evals actually are and why product teams should be…
@lennysan · Lenny Rachitsky on X♥ 968
My conversation with Head of ChatGPT and Codex, @thsottiaux We discuss: 🔸 Why the model picker is likely going away 🔸 Why loops and graphs are a passing phase 🔸 Why most actions on the internet will soon be taken by agents 🔸 OpenAI’s approach to AI safety Listen now 👇
@ttorres · Teresa Torres on X♥ 18
"I'm going to tell you my story because I think it's an amazing story of continuous improvement, of how teeny-tiny steps compound over time." Over the past year, my work has transformed completely—and it all started with a broken ankle and a willingness to get curious about AI. In this article, I…
@joulee · Julie Zhuo on X♥ 228
Too many teams are doing AI transformation poorly, so I wrote down everything I have learned in my journey and in chatting with many other leaders. A long read, so get your coffees.
@joulee · Julie Zhuo on X♥ 284
4 tests any personal AI needs to pass to go big: 1. The mom test 2. The spare key test 3. The putting-it-off test 4. The bored-in-line test ...and my review of how @Muse does across each of them! Buckle up with a coffee and a snack, cuz this one goes deep.
@ttorres · Teresa Torres on X♥ 9
💭 "I wish I could click on this card and be like, 'Clean this up.'" When a customer pointed out a flat, unstructured branch in her AI-generated opportunity solution tree, it sparked a three-week journey to fix the problem at its source. Here's what you'll learn from this article: How one customer…
@lennysan · Lenny Rachitsky on X♥ 469
My biggest takeaways from @thsottiaux: 1. Most actions on the internet will soon be taken by agents, and Tibo doesn’t think most people have priced this in yet. When Notion launched its MCP server, it saw a flood of traffic, which strained its systems and forced it to rethink its economics. Every…
@hnshah · Hiten Shah on X♥ 30
We asked Claude what changed among Linear's competitors. No skills: an old source treated as recent, and a guess stated as fact. Same model, with our competitive skills installed: every claim dated, the ranking marked as its own view, and a line about what it could not see.
@ttorres · Teresa Torres on X♥ 292
AI evals have been the "it" skill for product teams for over a year. I've even called evals a new discovery habit. But I still meet product teams who only have a vague idea of what evals are. And it's not their fault. Most of the writing on this topic is intended for engineers or just isn't…
@ttorres · Teresa Torres on X♥ 12
What does it take to build an AI app builder specifically for product managers—not engineers—inside an already crowded market? In this episode of Just Now Possible, Teresa Torres talks with Brian De Haaff (CEO and Co-Founder), Chris Waters (CTO and Co-Founder), and Sarah Moisan-Thomas (Senior…
@lennysan · Lenny Rachitsky on X♥ 1,133
Linear CEO @karrisaarinen: "I took a break this summer from paying attention to AI. I thought that when I came back, I'd feel left behind. But when I came back, I realized that nothing has really changed. There's new models, there's new techniques, there's new agents. But in the
@ttorres · Teresa Torres on X♥ 9
🎙️Delivery Isn’t Free Everyone's saying it: "Now that AI makes delivery free…" But is it? In this episode of All Things Product, Petra Wille and Teresa Torres pull apart one of the most repeated claims in product right now. They dig into why building a single feature has gotten dramatically…
@lissijean · Melissa Perri on X♥ 8
Most teams make the mistake of waiting for data to magically appear before building their AI product. They get stuck in the classic chicken-and-egg problem: you need data to build good AI, but you need users to generate that data. So they wait. And wait. And wait some more. /1
@ttorres · Teresa Torres on X♥ 17
Later this month I'll be hosting two mini-workshops on the skills that I think will differentiate the best product teams from the rest. AI Evals: The New Discovery Habit 🗓️ September 23, 2026, 9am-10:30am PDT In this session, you'll get introduced to what AI evals are, you'll receive a blueprint…
@lennysan · Lenny Rachitsky on X♥ 1,037
Trend I'm following: Teams using AI "simulations" to quickly test product ideas and flow tweaks. Products like @simile_ai, @primitivelabsai, @syntheticusers, and others. If you've tried something like this, how'd it go?
@lennysan · Lenny Rachitsky on X♥ 88
"Having to set up and fiddle with your loops… I don't think this is the way that it's going to work." @thsottiaux (Head of ChatGPT & Codex) says we are heading towards Dots being the primary way you talk to AI. Full conversation:
@hnshah · Hiten Shah on X♥ 35
We asked Claude what changed with Linear’s competitors. The first answer used an old source as if it were recent. It also stated a guess as fact. Then we gave the same model our competitive skills. It showed its limits up front. It explained how it ranked the changes and kept
@ttorres · Teresa Torres on X♥ 4
How do you build trustworthy AI diagnostic tools in one of medicine's most historically under-researched areas? In this episode of Just Now Possible, Teresa Torres talks with Tulsi Patel (Director of Product and Technology), Lorna Brightmore (Head of Data and AI), and Jack Pickard (Head of…
@ttorres · Teresa Torres on X♥ 5
What happens when you hand your opportunity solution tree to an AI? Vistaly rebuilt its entire product to find out—and the agents were the easy part. In this episode of Just Now Possible, Teresa Torres talks with Matt O'Connell (Co-Founder and CEO), CP Dehli (Co-Founder), and Steve Klein…
@hnshah · Hiten Shah on X♥ 24
AI makes bad product marketing look finished. That’s the part I don’t trust. A competitive brief can sound current while leaning on an old source. Positioning can sound sharp even when the logic underneath is a guess. A battlecard can look complete and still fail the rep who
@ttorres · Teresa Torres on X♥ 10
Product managers keep coming back to Aha! with the same request: "This prototype is great—now I want it to actually integrate with other systems and do real things." Chris (Aha!) explains how that moment—prototype meets production—reshaped their roadmap. Instead of handing the app to an engineer…
@ttorres · Teresa Torres on X♥ 11
Inspired by Git diff, Teresa Torres started simple: compare the input tree to the output tree and generate a change set. Then she learned the same pair of trees can produce multiple change sets—and only one tells the real story. A semantic merge means something very different from a delete plus…
@lissijean · Melissa Perri on X
This week on the podcast, @vlaurenlee from @Shopify shared how her team is using AI to solve AI problems, instead of waiting years for interaction data. "You're basically imparting all of your product intuition into an LLM to then shape yet another LLM." /2
@ttorres · Teresa Torres on X♥ 9
The supermarket shelf used to hold maybe 1,000 SKUs. Now it can hold 10,000, because AI makes it just as easy to build the wrong thing as the right one. Brian (Aha!) argues this is exactly where product managers earn their keep: strategy and discovery decide what deserves a spot on the shelf…