John Cutler examines the sudden enthusiasm for measuring "return on tokens" as AI usage grows inside companies. He argues that vendors and executives eager for clean numbers are reaching for yet another proxy, much like earlier fixations on hours, story points, and output counts. The piece questions whether these measures actually reflect delivered value or merely shift the gaming and distortion somewhere else. For product leaders, the stakes are high: choosing the wrong proxy can steer teams toward cheap-looking activity instead of outcomes customers feel. The essay invites readers to scrutinize what any measurement of AI-assisted work is really capturing.
01Key takeaways
- Be skeptical when a new easy-to-count metric suddenly becomes the headline measure of AI value.
- Ask what outcome a token or activity metric is meant to stand in for, and whether it actually tracks that outcome.
- Expect any proxy that becomes a target to be gamed, and watch for drift away from customer value.
- Anchor AI investment discussions in delivered outcomes rather than consumption or spend.
02Key sections
- The rush toward return on tokens
- Vendors and companies alike are embracing token-based ROI, partly because they have struggled to measure product value in the first place. Cutler notes that the appeal is strongest where leaders hoped AI spend would offset headcount costs.
- A long history of proxies
- The essay places token metrics alongside hours, story points, and other activity measures that organizations have long used as stand-ins for value. Each one tends to become a target and then a distortion.
- Why proxies mislead
- Cutler argues that when a proxy is easy to count, people optimize for it rather than for the underlying outcome. The measure gradually drifts away from the thing it was supposed to represent.
03From the post
“Everyone is talking about “return on tokens.” Vendors love it (as long as the news is good). Companies that had long since given up on any sort of value architecture or on understanding the ROI of product work are suddenly chomping at the bit. Which makes sense: they haven’t been able to lay off enough people to cover their token…”
Summary and takeaways written by PM Atlas; quotes are short excerpts. © the original author.