Product Talk · Free post · Discovery & customer research · Metrics, data & experimentation

Creating a Sustainable Continuous Discovery Practice at CarMax [Case Study]

Teresa TorresSep 12, 20188 min
SourceProduct Talk
KindFree post
PublishedSep 12, 2018
Originalproducttalk.org ↗
N:

Teresa Torres describes coaching the CarMax Digital Merchandising team, a mature product team that already practiced discovery but lacked deliberate organization. Through interviews with the PM, designer, and tech lead, the piece shows how the team moved from periodic customer contact and big single experiments to a continuous cadence of customer conversations, assumption-focused experiments, and an opportunity solution tree. It matters because it offers a concrete picture of what a sustainable continuous discovery habit looks like inside an established company, not just a startup.

01Key takeaways

  • Keep customer conversations on a steady weekly cadence, not only when you have something specific to validate.
  • Design experiments around one assumption at a time rather than testing every risk in one large experiment.
  • Treat any method that gathers evidence on an assumption as an experiment, including interviews and prototype tests.
  • Test for desirability before usability: confirm people want the solution before optimizing how it works.
  • Use a living opportunity solution tree to keep the whole team's thinking visible and prevent jumping to solutions too early.
  • Agree on success thresholds with stakeholders before running an experiment so results lead to clear next steps.

02Key sections

The team and its starting point
The CarMax team already had solid discovery habits and a backlog of opportunities, but their process was loosely organized. Their customer contact was sporadic and interviews produced lots of notes without clear takeaways.
Shifting to continuous customer contact
The team moved from talking to customers only when they had something to validate to meeting 2-4 customers weekly. They also built productivity habits that made recruiting and interviewing easier.
Redesigning experiments around assumptions
Rather than one large test covering every risk, the team began testing individual assumptions and treating any data-gathering method as an experiment. This let them run many more tests in parallel and learn faster.
Using the opportunity solution tree
The team adopted the opportunity solution tree as a living visual of their focus and options. It disciplined them to deliberate on opportunities before jumping to solutions, which was especially valuable for the engineer on the team.
Results and advice for other teams
The team reports better customer alignment, faster and more confident decisions, and a practice of testing desirability before usability. They recommend working with a coach who tailors the program and builds lasting habits.

03From the post

“Back in March, I shared a case study about one of the product teams I worked with at Snagajob (now Snag). I’m excited to share another one with you today. This case study is about a team at CarMax that I just finished working with. You’ll see that”

“Designing experiments around individual assumptions vs. trying to test everything with one large experiment helped us iterate quicker and move towards what the customer values…”Teresa Torres · Product Talk
“The opportunity solution tree is a living diagram that shows where we are focused and where our opportunities could take us.”Teresa Torres · Product Talk
“Now that we're more methodical and structured in identifying opportunities and solutions, we're able to "fail fast" or iterate forward in a much more informed…”Teresa Torres · Product Talk

04Frameworks mentioned

Summary and takeaways written by PM Atlas; quotes are short excerpts. © the original author.