Search: Hamel Husain

Lenny’s Podcast131 min

Why AI evals are the hottest new skill for product builders

Hamel Husain & Shreya Shankar · Sep 25, 2025

Key learnings

Evals are systematic ways to measure and improve an AI application, essentially data analytics on LLM behavior, replacing…
Start with error analysis by manually reviewing around 100 sampled traces and writing short notes on the first upstream failure…
Product people with domain expertise should lead open coding; appoint one 'benevolent dictator' whose judgment you trust instead…

Watch ↗

Product Talk · Teresa TorresOct 2, 2025

Debugging AI Products: From Data Leakage to Evals with Hamel Husain

Oct 2, 2025

Teresa Torres and Hamel Husain discuss how to evaluate and debug AI products with a scientist's mindset. Hamel draws on his machine learning background at Airbnb and GitHub, and his consulting work with startups like…

Read at producttalk.org ↗

Lenny’s NewsletterSep 9, 2025

Building eval systems that improve your AI product

Sep 9, 2025 · 20 min

Subscriber post — summary only

Hamel Husain and Shreya Shankar outline a practical playbook for building AI evaluation systems that improve a product rather than just producing dashboards. The core argument is to start with error analysis by a domain…

Read at lennysnewsletter.com ↗

“Hamel Husain”

Why AI evals are the hottest new skill for product builders

Debugging AI Products: From Data Leakage to Evals with Hamel Husain

Building eval systems that improve your AI product