Listen to the original at Lenny’s Podcast ↗lennysnewsletter.com
By Lenny Rachitsky · with Ramesh Johari · lennysnewsletter.com · @lennysan on X · YouTube · LinkedIn
Ramesh Johari is a Stanford professor and startup advisor who has worked with marketplace companies including oDesk/Upwork and Airbnb-era teams on data science and experimentation. The conversation covers what marketplaces really sell (removing transaction costs), common founder mistakes, how data scientists should focus on decisions rather than predictions, experimentation pitfalls, and the design of rating systems.
01Key learnings
- Marketplaces sell the removal of friction (transaction costs) rather than the underlying goods, and both sides are customers who depend on the platform to reduce that friction.
- Don't build a marketplace before you have one: early on, find a bespoke value proposition that solves a real problem (e.g., UrbanSitter's credit card payments, oDesk's remote work trust tools) without needing scaled liquidity.
- Apply a scaled-liquidity smell test: if you don't have a lot of buyers and sellers, you're not yet a marketplace; if one side is scaled, use it to attract the other side.
- Be wary of early monetization commitments such as a flat take rate; as relationships mature, platforms risk disintermediation and need pricing that reflects ongoing value.
- Data scientists should focus on helping the business make decisions, which requires causal thinking about differences in outcomes, not just predicting correlations like who is most likely to convert or be hired.
- Evaluate ranking or matching algorithms by downstream business outcomes (bookings, rehires, ratings) through experiments, not by how well they recreate past choices.
- Marketplace changes are often whac-a-mole: reallocating attention creates winners and losers, so judge changes by whether the gained winners matter more to the business than the losers lost.
- Treat experiments as learning, not just winners and losers; cultivate incentives and culture that reward informative failures, and consider Bayesian approaches that carry prior knowledge into future tests.
“Marketplaces are a little bit like a game of whac-a-mole.”Ramesh Johari · Lenny’s Podcast · 00:00:00
“What's the moral there? The moral is a marketplace business never starts as a marketplace business.”Ramesh Johari · Lenny’s Podcast · 00:14:50
“Prediction is inherently about correlation. But when we ask people to make decisions, we're asking them to think about causation.”Ramesh Johari · Lenny’s Podcast · 00:34:50
02Frameworks mentioned
Scaled liquidity smell testMarket failures and transaction costsCausal inference versus machine learning predictionBayesian A/B testing with priors
Summary and takeaways written by PM Atlas; quotes are short excerpts. © the original author.