Aishwarya Naresh Reganti + Kiriti Badam

Aishwarya Naresh Reganti and Kiriti Badam are AI practitioners with deep experience at companies like Amazon, Google, Databricks and OpenAI, and they co-teach a popular course on building AI products. The conversation covers why AI products differ from traditional software (non-determinism and the agency-control trade-off), a step-by-step approach to increasing autonomy, evaluation practices, and the continuous calibration, continuous development framework.

01Key learnings

• AI products differ from traditional software mainly because user inputs and model outputs are both non-deterministic, so behavior must be anticipated and calibrated rather than fully specified.
• Each time you grant an AI system more decision-making autonomy, you give up some human control; autonomy should be earned through demonstrated reliability.
• Start with low-agency, high-human-control versions (e.g., suggestions to human agents) and log human corrections to build a feedback flywheel before moving toward full automation.
• Begin with a problem-first mindset: break the problem into autonomy levels so you focus on the user's actual need instead of solution complexity.
• Combine evals (pre-deployment test datasets encoding product judgment) with production monitoring using implicit signals like regenerations; neither alone catches every failure.
• Leaders need to stay hands-on and relearn their intuitions about AI, since prior experience may not hold and bottom-up adoption depends on top-down understanding.
• Be skeptical of 'one-click' autonomous agents; reliable enterprise AI typically takes months of workflow understanding, data cleanup and iteration.
• As implementation gets cheaper, prioritize judgment, taste, problem understanding and design, and treat accumulated hard-won learning ('pain') as a durable moat.

02Frameworks mentioned

Summary and takeaways written by PM Atlas; quotes are short excerpts. © the original author.