What happens when you hand your opportunity solution tree to an AI? Vistaly rebuilt…

What happens when you hand your opportunity solution tree to an AI? Vistaly rebuilt its entire product to find out—and the agents were the easy part. In this episode of Just Now Possible, Teresa Torres talks with Matt O'Connell (Co-Founder and CEO), CP Dehli (Co-Founder), and Steve Klein (Co-Founder) at Vistaly, a company Teresa partners with and builds for. V1 was a purpose-built canvas for continuous discovery. V2 is a ground-up rewrite: upload customer interviews, get a snapshot of each, and let an agentic workflow draft and update your tree. They get specific about what broke. A chat-based synthesis experiment users found too slow. A "house of cards" problem where one badly framed opportunity corrupts every layer above. Teresa walks through the AI evals work behind one customer complaint—"I'll have to come back and clean up this branch"—including an LLM as a judge she couldn't calibrate and the orchestration change that fixed what no prompt could. You'll hear why change sets had to be taught to the agent as the rules of a game instead of diffed at the end, how data residency pushed them onto Bedrock, and why their hardest problem isn't output quality—it's helping people comprehend what changed. Guests - Matt O'Connell – Co-Founder and CEO, Vistaly - CP Dehli – Co-Founder, Vistaly - Steve Klein – Co-Founder, Vistaly - Teresa Torres – host of Just Now Possible, building the AI synthesis services for Vistaly V2 Key Takeaways - The competition isn't good interview synthesis done faster—it's shallow synthesis that never really happened. That changes what "good enough" means. - Layered analysis matters more than whether a step is a pipeline or an agent. If the interview snapshot is wrong, every layer above it inherits the error. - Prompt changes run out. Balancing two opposing error modes took an orchestration change—moving the check into an agentic repair loop—not a better prompt. - Change sets can't be diffed at the end. The same input/output tree pair has multiple valid change sets, and only the semantically meaningful one makes sense to a user. - Users don't want to collaborate with the AI step by step. They want the answer, then the ability to correct it. In this episode - Why Vistaly rebuilt from scratch instead of layering AI features onto V1—and shut off V1 signups to protect the rewrite - Rebuilding nearly all of V1's functionality in two and a half months, after three years of building it the first time - The V1 chat interface that walked users through insights one at a time, and why "still faster than before" wasn't fast enough - How V2 works: upload three interviews, generate an interview snapshot for each, then synthesize a first-draft opportunity solution tree - The "house of cards" problem: garbage in at the snapshot layer corrupts the tree above it - Using a code assertion (too many children on a node) as a cheap pre-filter before spending money on an LLM as a judge - Four new evals and 16 experiment variants to break a seesaw between missing subgroupings and badly framed parents - Why the fix was a repair loop in the orchestration, not a better prompt—and how that eval became a production guardrail - Teaching the agent "the rules of the game" so it logs semantic moves (merge, move, reframe) and produces reliable change sets - Stable IDs, data provenance, and the "basic computer science" Teresa swept under the rug in the prototype - What beta data revealed about interview quality: sales demos, stakeholder meetings, and LLM-generated fake transcripts uploaded as "interviews" - Building a 40-point system for generating realistic synthetic transcripts as test data - Driving tree edits through MCP, including mapping a full Jira epic backlog into the opportunity space - Adding an agent back to the canvas, and the versioning and rollback problem that comes with it - Data residency, SOC 2, GDPR, and moving to European Bedrock inference - Why model upgrades aren't drop-in: prompts are model- and version-specific, and Bedrock constrains tokens independently of the model Resources & Links - Vistaly: — Opportunity solution tree software with AI-generated interview snapshots and tree synthesis - Product Talk: — Teresa's AI evals guide and her blog posts on building the Vistaly synthesis services - Model Context Protocol: — How Vistaly users drive tree edits from outside the product - AWS Bedrock: — Where Vistaly runs inference, including EU-hosted inference for European customers - Just Now Possible episode with Banani: — the canvas-plus-agent design UX Teresa references - Just Now Possible episode with Zencity: — the layered data pyramid Teresa references - All Things Product episode on the quality of evidence, with Petra Wille: Chapters 00:00 Meet the Founders 01:05 How Vistaly Began 03:10 Community First Partnership 04:56 Why Build OST Software 07:18 OST Explained Fast 10:28 Evidence Inside the Tree 12:56 AI Changes Discovery 16:58 Chat UI Experiment 23:00 Should AI Do Synthesis 27:10 Rebuilding as V2 32:08 Turning Off V1 Signups 34:43 V2 Workflow and Tensions 39:35 Trusting AI Outputs 39:56 Layered Analysis Pyramid 41:25 Building Better Trees 43:34 Bad Inputs and Coaching 47:20 Teaching Through Software 50:12 Evals and Guardrails 55:23 Deterministic Change Sets 01:00:54 UX for Tree Updates 01:07:12 Chat Agents and Versioning 01:11:17 Privacy Compliance Tradeoffs 01:15:06 Model Selection Reality 01:17:32 What Comes Next 01:19:08 Closing and Links Listen on Spotify, Apple Podcasts, or watch on YouTube. Spotify: Apple Podcasts: YouTube:Teresa Torres on X · Oct 1, 2026