Drawer 11 · 318 notes · 58 people · 2005–2026

Metrics, data & experimentation

North-star metrics, analytics, A/B tests, and data-informed decisions

Q:
Answers from Metrics, data & experimentation notes, cited
@lissijean · Melissa Perri on X♥ 1
This is what separates teams that accelerate with AI from those that get stuck. You don't find perfect data lying around. You create it using the deep product intuition you already have. /end Check out the whole episode with @vlaurenlee here:
Lenny’s Podcast110 min

Aishwarya Naresh Reganti + Kiriti Badam

Aishwarya Naresh Reganti + Kiriti Badam · Jan 11, 2026
Key learnings
  • AI products differ from traditional software mainly because user inputs and model outputs are both non-deterministic, so behavior…
  • Each time you grant an AI system more decision-making autonomy, you give up some human control; autonomy should be earned through…
  • Start with low-agency, high-human-control versions (e.g., suggestions to human agents) and log human corrections to build a…
Lenny’s Podcast145 min

5 questions to ask when your product stops growing

Jason Cohen · Jan 25, 2026
Key learnings
  • Compute your growth ceiling: new customers added per month divided by monthly cancellation rate gives the maximum customer count…
  • Ask 'what made you cancel?' rather than 'why did you cancel?' and use open-ended responses, since multiple-choice lists skew…
  • Be skeptical of 'too expensive' as a cancellation reason; customers who already bought didn't find the price too high, so dig…
Lenny’s Newsletter · Free post♥ 181

Product-led marketing

Mar 28, 2023 · 15 min

Kyle Poyar argues that product-led growth (PLG) only works economically if customer acquisition cost stays near zero, roughly under $1 per unique website visitor. Using OpenView benchmark funnel math, he shows that a…

Lenny’s Podcast81 min

OpenAI researcher on why soft skills are the future of work

Karina Nguyen · Feb 9, 2025
Key learnings
  • Build rigorous evals first: define deterministic pass/fail behaviors (e.g., correct time handling) and human win-rate comparisons…
  • Use prompting as a fast prototyping method to test product ideas before engineering investment, as Karina did with file uploads…
  • Synthetic data scales cheaply for teaching core product behaviors, but human expert data is still needed for specialized…
Product Talk · Teresa TorresNov 21, 2011

What Product Assumptions Are You Making?

Nov 21, 2011 · 6 min

Teresa Torres argues that product teams routinely build on untested assumptions, treating hopes as facts, and that this risks building something nobody wants. Using an imagined Caltrain commuter app, she shows how to…

@lissijean · Melissa Perri on X♥ 2
They're taking everything they know about commerce, customer behavior, and product strategy, and using that knowledge to systematically generate the training data they need. It transforms the data bottleneck from a passive waiting game into an active engineering challenge. /3
Lenny’s Newsletter♥ 44

Winning at SEO

Jul 14, 2020 · 14 min
Subscriber post — summary only

A guest-written post in which an SEO leader lays out a strategy for an early-stage company to compete with entrenched incumbents: build many valuable, templated pages targeting long-tail keywords, using proprietary…

Lenny’s Podcast101 min

Building Anchor, selling to Spotify, and lessons learned

Maya Prohovnik · Sep 28, 2023
Key learnings
  • Dogfood your product by creating content with it yourself, since building creator tools without living the creator experience…
  • Ensure the team experiences the product's pain directly, for example by having engineers and PMs make their own podcasts, to…
  • Balance data and gut instinct by treating your intuition as a legitimate data point, explaining its reasoning with evidence…
Lenny’s Newsletter♥ 25

Breaking into growth

Nov 2, 2021 · 7 min
Subscriber post — summary only

This post is mostly a promotional announcement for a part-time PM course aimed at early-career product managers, with a reader Q&A in the middle. The core advice in the Q&A explains how to move from a generalist PM role…

Product Talk · Teresa TorresJan 2, 2014

My Favorite Tools of the Trade

Jan 2, 2014 · 5 min

Teresa Torres shares the personal productivity, reading, product-building, and collaboration tools she relies on as a product manager, startup builder, and graduate student. She opens with a caveat that she once…

SVPG · Marty CaganSep 1, 2020

Discovery – Learning vs. Insights

Sep 1, 2020 · 3 min

Marty Cagan, who popularized the term product discovery, clarifies a common confusion: learning is valuable, but the real goal of discovery is to gain insights, which are learnings a team can act on. He explains that…