Drawer 11 · 316 notes · 58 people · 2005–2026

Metrics, data & experimentation

North-star metrics, analytics, A/B tests, and data-informed decisions

Q:
Answers from Metrics, data & experimentation notes, cited
Lenny’s Podcast102 min

Al Engineering 101 with Chip Huyen (Nvidia, Stanford, Netflix)

Chip Huyen · Oct 23, 2025
Key learnings
  • Focus on what actually improves AI apps, such as talking to users, improving data, writing better prompts, and optimizing…
  • Be cautious about committing to newly released technologies that have not been widely tested, since switching away from them…
  • Fine-tuning and post-training can shape model behavior a lot, and many teams now focus effort there because base pre-training…
Lenny’s Podcast74 min

How to grow a subscription business

Yuriy Timen · Sep 1, 2022
Key learnings
  • Give every channel a real test before judging it; many teams wrongly conclude YouTube or other tactics don't work because they…
  • Virality works best when the product has inherent network effects, where utility grows with more users, rather than trying to…
  • Treat SEO as a 3-month time-boxed experiment, favoring angles with a unique editorial, programmatic, or data advantage, and buy a…
Lenny’s Newsletter · Free post♥ 84

How To Know If You're Supply or Demand Constrained 🤹‍♂️ - Phase 2 of Kickstarting and Scaling a Marketplace Business

Dec 10, 2019 · 9 min

““Lyft was always supply constrained. It was obvious to everyone. You open up the app on a Friday night and there were no cars available. Instead, you got a Lyft message saying sorry, try again later. We eventually ended up having to build a waitlist for supply to catch up to demand. This was especially bad for us because Lyft focused on being reliable — a ride when you need one, with the tap of a button. The aha moment, or inflection…”

SVPG · Marty CaganJun 1, 2011

The Two Core Competencies

Jun 1, 2011

Marty Cagan argues that good product teams must excel at product discovery, which is fundamentally a skill of learning quickly: identifying the right target customer, finding the key problems to solve, and designing…

Lenny’s Newsletter♥ 86

What is good monthly churn

Feb 8, 2022 · 10 min
Subscriber post — summary only

Lenny Rachitsky benchmarks what counts as good and great monthly SaaS churn, drawing on growth leaders and ProfitWell data across B2C, SMB/mid-market, and enterprise segments. He argues churn should be read by price…

SVPG · Marty CaganApr 15, 2012

Defining Good

Apr 15, 2012 · 3 min

Marty Cagan responds to a common challenge: why are products like Facebook, Amazon, and Google so successful if they seem poorly designed? He argues that product people often confuse their own tastes with the target…

Lenny’s Newsletter · Free post♥ 105

Six rules of hiring for growth

Nov 9, 2021 · 15 min

Elena Verna argues that a founder's first growth hire is one of the most consequential hires a startup makes, and that many founders begin with avoidable mistakes. She proposes six rules: define a growth model before…

SVPG · Marty CaganJun 16, 2012

Lessons From Leaders

Jun 16, 2012 · 3 min

Marty Cagan shares a curated list of favorite product and leadership mantras he has collected while visiting companies, explaining what each one reveals about good product practice. Many of the quotes are attributed to…

SVPG · Marty CaganOct 13, 2011

Lean Thinking

Oct 13, 2011 · 5 min

Marty Cagan reviews Eric Ries's book "The Lean Startup" and argues it is one of the best product books in years, relevant to both new and experienced product people and not only early-stage startups. He notes Ries…

Lenny’s Podcast93 min

Lessons from scaling Ramp

Sri Batchu · Jun 25, 2023
Key learnings
  • Sequence B2B growth roughly as founder-led sales, first salespeople, low-cost targeted marketing like content and community, then…
  • Use early investor and founder relationships as a growth channel: getting influential operators onto the cap table can drive word…
  • Make every growth channel driven by technology and data, such as dedicated growth engineers automating sales workflows and…
Lenny’s Podcast119 min

The one question that saves product careers

Matt LeMay · Aug 14, 2025
Key learnings
  • Ask whether you would fully fund your own team if you were the CEO; if you can't answer confidently, that signals a risk worth…
  • Set team goals no more than one step away from company goals, so leadership immediately understands how your work contributes to…
  • Keep impact first at every step, from OKRs and strategy to epics and daily backlog, rather than letting goals get cascaded into…
Lenny’s Podcast133 min

An inside look at X’s Community Notes

Keith Coleman & Jay Baxter · Feb 27, 2025
Key learnings
  • Surface notes only when raters who normally disagree both find them helpful; seeking cross-polarized agreement is what keeps…
  • Open participation to everyone with basic verification and random selection, rather than curating contributors, so the system…
  • Set the quality bar conservatively: showing a bad note damages trust far more than withholding a good one, so the team accepts…