Drawer 11 · 316 notes · 58 people · 2005–2026

Metrics, data & experimentation

North-star metrics, analytics, A/B tests, and data-informed decisions

Q:
Answers from Metrics, data & experimentation notes, cited
Lenny’s Newsletter · Free post♥ 172

How to measure cohort retention

Aug 30, 2022 · 18 min

Olga Berezovsky, guest-writing for Lenny Rachitsky, argues that cohort retention is widely misunderstood and costly to get wrong, and offers a practical guide to measuring it. She recommends defining "active" by the…

Lenny’s Newsletter · Free post♥ 1,022

How Duolingo reignited user growth

Feb 28, 2023 · 21 min

Jorge Mazal, former CPO of Duolingo, recounts how the company reversed a slide in daily active user growth and eventually grew DAU about 4.5x over four years. The journey began with failed attempts: a…

Lenny’s Newsletter · Free post♥ 108

Choosing Your North Star Metric

Jun 16, 2021

“This is just quick email to let you know I wrote a guest post for Future (a16z’s new media site) that I’m pretty excited about. It’s a guide for helping you choose your North Star Metric(s), based on a survey of current and past employees at over 40 of today’s most successful growth-stage companies. Check out the table below for a glimpse of what you’ll find. Your North Star Metric is your strategy, and your strategy is your North Star…”

Lenny’s Newsletter♥ 13

When NOT to run an experiment – Issue 54

Dec 1, 2020 · 7 min
Subscriber post — summary only

The post argues that experiments are usually the right default for product changes, but describes when to skip them. It covers the benefits and downsides of A/B testing, then three cases to ship without testing: when…

SVPG · Marty CaganFeb 15, 2016

Ten Keys to Product Optimization

Feb 15, 2016 · 6 min

Cagan argues that product optimization, typically A/B testing and iterative tuning, is often confused with product discovery, and that optimization only makes sense after a team has found product/market fit. He…

Product Talk · Teresa TorresSep 18, 2012

Measuring The Impact Of Each User Story

Sep 18, 2012 · 3 min

Teresa Torres argues that choosing product themes is not enough; teams also need a way to predict and measure how each user story moves the product goal. Using an event-site example with the goal of increasing the…

Product Talk · Teresa TorresJun 17, 2013

Read Lean Analytics

Jun 17, 2013 · 3 min

Teresa Torres reviews Lean Analytics by Alistair Croll and Ben Yoskovitz and argues it is an essential, accessible guide to product and startup analytics. She notes it covers the basics of good metrics and segments…

SVPG · Marty CaganMay 1, 2011

Flying Blind

May 1, 2011 · 3 min

Marty Cagan argues that many product teams still fly blind because they do not instrument their products to collect usage analytics, or collect so little that they cannot see how customers actually use features. He…

Product Talk · Teresa TorresMay 14, 2013

Measure What Matters

May 14, 2013

Teresa Torres recounts advising a junior designer who wanted to grow in user experience design. She argues that hands-on learning matters more than reading or talking, and that the designer's best growth opportunity is…

Lenny’s Newsletter♥ 26

Fostering a culture of experimentation

Sep 22, 2020 · 11 min
Subscriber post — summary only

Lenny answers a reader question by unpacking how Airbnb built a strong experimentation culture, using its early data science hires, north-star metric, embedded data scientists, and trustworthy tooling as the core story…