Drawer 11 · 316 notes · 58 people · 2005–2026

Metrics, data & experimentation

North-star metrics, analytics, A/B tests, and data-informed decisions

Q:
Answers from Metrics, data & experimentation notes, cited
SVPG · Marty CaganJul 12, 2007

Make a Friend in Finance

Jul 12, 2007

Cagan argues that many product managers, especially those coming from engineering, have only a shallow grasp of how their product and company make money. He recommends cultivating a relationship with someone in the…

Lenny’s Podcast76 min

Mastering onboarding

Lauryn Isford · Feb 12, 2023
Key learnings
  • Experiment when you need to, mainly as risk mitigation for big changes; let customer research and mockups provide conviction for…
  • Onboarding is a critical choke point for self-serve products because downstream conversion, deal closing, and team-wide adoption…
  • For complex products, an immersive guided wizard that reduces cognitive load can outperform tooltips; tooltips still have a place…
Lenny’s Podcast90 min

Inside Etsy’s product, growth, and marketplace evolution

Tim Holley · Sep 3, 2023
Key learnings
  • Make one north-star KPI, such as gross merchandise sales, the drumbeat of every meeting and the yardstick for judging launches.
  • Repeat the company narrative relentlessly; Tim suggests people need to hear it several more times than you'd expect before they…
  • Benchmark against the broader set of places buyers spend money, not just direct marketplace competitors, to ground prioritization…
SVPG · Marty CaganMay 18, 2012

Product Scorecard Stages

May 18, 2012 · 4 min

Marty Cagan argues that a product team's primary scorecard metric should change depending on the stage the product is in. He describes three stages: MVP, where the only goal is winning several happy, referenceable…

SVPG · Marty CaganMar 17, 2025

Outcomes Are Hard

Mar 17, 2025 · 10 min

Marty Cagan and Felipe Castro argue that moving from outputs (what we build) to outcomes (the difference we make) is conceptually simple but practically very hard. They contend that most failed attempts stem from trying…

Product Talk · Teresa TorresFeb 2, 2026

Ch. 3: Focusing on Outcoes Over Outputs

Feb 2, 2026 · 6 min

“Continuous Discovery Habits turns five this year. And to celebrate we are reading the book together. Each month, I am releasing an in-depth reading guide that includes: * The chapters we will be reading * A preview of the most important concepts we'll be learning about * Short videos you can”

Lenny’s Podcast93 min

Developing a growth model + marketplace growth strategy

Dan Hockenmaier · Oct 9, 2022
Key learnings
  • A growth model is a spreadsheet formula for how your business grows; roughly half its value comes from the act of building it…
  • Start with three core building blocks for a SaaS growth model: acquisition channels, retention, and monetization; marketplaces…
  • Retention is often the most sensitive lever in a growth model, so small retention gains can be worth more than larger gains…
Lenny’s Newsletter · Free post♥ 192

Pricing your SaaS product

Oct 27, 2020 · 18 min

Patrick Campbell, CEO of ProfitWell, argues that pricing is an ongoing strategic lever rather than a one-time number, and that early on the exact price matters far less than the value metric and the customer segments…

Lenny’s Podcast124 min

The ultimate guide to performance marketing

Timothy Davis · Jul 28, 2024
Key learnings
  • Paid acquisition suits most businesses because paid search is user-driven, so start by checking where users already find you and…
  • Run small signs-of-life tests first, starting with your own customer data for lookalike audiences at a 1% match, before…
  • Don't copy winning creative across platforms; user mindset and behavior differ, so adapt messaging for each platform.
SVPG · Marty CaganFeb 20, 2011

Product Discovery With Live-Data Prototypes

Feb 20, 2011 · 5 min

Cagan argues that the core goal of product discovery is finding the smallest valuable, usable, feasible product as fast as possible by failing fast. He describes two techniques: disposable user prototypes tested with…

SVPG · Marty CaganMay 31, 2006

Market Research Capabilities and Limitations

May 31, 2006 · 7 min

Marty Cagan argues that market research tools such as surveys, analytics, focus groups, site visits and usability testing are valuable but have clear limits in product work. Tensions between marketing and product teams…

Lenny’s Podcast115 min

How to find hidden growth opportunities in your product

Albert Cheng · Oct 5, 2025
Key learnings
  • Alternate between exploring for new insights and exploiting proven wins; when experiments stop reaching significance, return to…
  • Share experiment learnings across the company so adjacent teams can apply the same human-psychology insight to their own parts of…
  • Make a free tier a taste of the full product's value rather than a stripped-down version; sampling paid suggestions to free users…
Lenny’s Podcast69 min

An inside look at Mixpanel’s product journey

Vijay · Jan 26, 2023
Key learnings
  • Protect your core product: rather than moving people from the core to new ventures, invest the core's profits into new bets…
  • Be cautious adding adjacent products in new categories; bolt-on products that are only mid-tier in crowded categories rarely move…
  • When a product is losing customers to competitors, a focused 'speed' phase working directly from ranked churn reasons can close…
Lenny’s Podcast97 min

The 100-person AI lab that became Anthropic and Google's secret weapon

Edwin Chen · Dec 7, 2025
Key learnings
  • Define quality in rich, specific terms for each domain (e.g., what makes a poem Nobel-worthy, not just whether it has eight…
  • Gather thousands of signals on each worker and task, from background and expertise to performance, and use them like an ML…
  • Don't trust benchmarks at face value: many contain wrong answers, and optimizing for them can make models worse at messy…
SVPG · Marty CaganAug 9, 2014

Profit and Loss Responsibility

Aug 9, 2014

Marty Cagan addresses a recurring question from company leaders: should product managers carry P&L responsibility? He traces the idea to consumer packaged goods, where a product manager owned costs, marketing, forecasts…

Product Talk · Teresa TorresSep 21, 2026

Ch. 10: Testing Assumptions

Sep 21, 2026 · 5 min

This reading guide kicks off a monthly book club celebrating five years of Continuous Discovery Habits, focused this month on Chapter 10, 'Testing Assumptions.' The guide frames the chapter as a bridge between…

SVPG · Marty CaganJan 31, 2006

Rapid Response

Jan 31, 2006 · 3 min

Marty Cagan argues that many teams disband the people who built and launched a product right after release, wasting the period when learning and correction are most valuable. He proposes a dedicated "Rapid Response"…

Lenny’s Podcast118 min

Inside Google's AI turnaround: The rise of AI Mode, strategy behind AI Overviews, and their vision for AI-powered search

Robby Stein · Oct 10, 2025
Key learnings
  • Search demand is expanding rather than dying: AI lets people ask more and harder questions, so growth comes from new query types…
  • Build a product from a real user problem and stay dissatisfied with the status quo; keep making it better until it tips into…
  • Use a J-curve retention analysis (day 7, 30, 90) to tell whether a product is flattening into real usage or steadily losing users.
Product Talk · Teresa TorresDec 10, 2014

Don't Rely On Confidence Alone

Dec 10, 2014 · 3 min

Teresa Torres argues that confidence in a product idea reflects how coherent a story feels in our heads, not whether the idea is true. She points to Research in Motion and AOL as examples of confident strategies that…