Listen to the original at Lenny’s Podcast ↗youtube.com
By Lenny Rachitsky · with Keith Coleman & Jay Baxter · lennysnewsletter.com · @lennysan on X · YouTube · LinkedIn
Keith Coleman (VP of Product) and Jay Baxter (ML lead) are the product and machine-learning leaders behind X's Community Notes, a crowdsourced system for adding context to potentially misleading posts. The conversation covers how the bridging-based ranking algorithm works, the team's operating model and principles, how the product survived multiple leadership changes, and where it is headed.
01Key learnings
- Surface notes only when raters who normally disagree both find them helpful; seeking cross-polarized agreement is what keeps notes neutral, accurate, and hard to manipulate.
- Open participation to everyone with basic verification and random selection, rather than curating contributors, so the system feels fair and trustworthy.
- Set the quality bar conservatively: showing a bad note damages trust far more than withholding a good one, so the team accepts showing only about 8% of proposed notes.
- Make the product prove itself at each stage with data before expanding, which built internal confidence and eased skepticism from trust and safety teams.
- Run a small, fully focused team with one clear owner and one senior decision-maker, using light tools like a shared doc instead of heavy task management and OKRs.
- Keep core principles like transparency, open-sourced code and data, and no manual override button, since breaking them would make the product harder to trust.
- Allowing pseudonymous contributors surfaced more honest cross-partisan agreement than real-name identities, a counterintuitive finding worth testing in your own community features.
- Deleting unnecessary code and features is as valuable as shipping new ones, because maintenance burden from small incremental wins quietly accumulates.
“We actually look for agreement from people who have disagreed in the past.”Jay Baxter · Lenny’s Podcast · 00:00:18
“We view the worst possible mistake as showing a bad note because that's going to undermine trust and the trust is why people like the…”Keith Coleman · Lenny’s Podcast · 00:20:23
“Working on this product has made me realize just how many things I used to trust by default, that now I look at more skeptically.”Keith Coleman · Lenny’s Podcast · 00:26:18
02Frameworks mentioned
Summary and takeaways written by PM Atlas; quotes are short excerpts. © the original author.