OpenAI researcher on why soft skills are the future of work

Karina Nguyen is an AI researcher who has worked on post-training and evaluation at Anthropic (Claude 3, file uploads with 100K context) and at OpenAI (Canvas, tasks, o1). The conversation covers how models are trained and debugged, synthetic data and evals, how product features like Canvas were built through rapid iteration, and which human skills may matter most as AI advances.

01Key learnings

• Build rigorous evals first: define deterministic pass/fail behaviors (e.g., correct time handling) and human win-rate comparisons to measure progress and catch regressions (Karina Nguyen).
• Use prompting as a fast prototyping method to test product ideas before engineering investment, as Karina did with file uploads and personalized starter prompts.
• Synthetic data scales cheaply for teaching core product behaviors, but human expert data is still needed for specialized knowledge; use real user feedback to shift distributions after beta.
• Treat model debugging like software debugging: trace conflicting training signals (e.g., body-awareness vs. function calls) that cause confused or over-refusing behavior.
• Design AI features around the form factor users already know (reminders, documents, notifications) and add the AI-specific magic on top, as with tasks and Canvas.
• Invest in creative ideation, listening to users, and rapid iteration, which Karina argues remain hard for models to replicate.
• Prioritization and people skills such as empathy, communication and collaboration matter greatly, since research progress is bottlenecked by choosing which bets get compute.
• Build for where models are going, not just current capability: ideas that become strong once models improve can win over time.

02Frameworks mentioned

Summary and takeaways written by PM Atlas; quotes are short excerpts. © the original author.