Product Talk · Free post · Execution, roadmaps & process · Metrics, data & experimentation

Weekend Reading: A/B Testing Is Hard, To Pivot Or Not, And More

Teresa TorresFeb 1, 2013
SourceProduct Talk
KindFree post
PublishedFeb 1, 2013
Originalproducttalk.org ↗
N:

This is a weekly roundup of product-building reads rather than a single argument. It pulls together a few threads: deciding what to ignore so you can focus on the most critical work, the primacy of conversion strategy over individual test tactics, the statistical pitfalls that undermine A/B tests, how to evaluate whether a product pivot is working, and the value of building a support network. Because each pick is recommended in a sentence or two, the piece works mainly as a curated reading list. It matters for product managers because it points toward durable habits in prioritization, experimentation rigor, and founder resilience.

01Key takeaways

  • Assume you will never finish everything and identify the single most critical piece to build first.
  • Choosing what to ignore is a strategic decision, not just a time-management one.
  • Prioritize overall conversion strategy over isolated optimization tactics.
  • Learn basic statistics so you can tell which A/B test results are actually trustworthy.
  • Establish clear criteria for evaluating whether a product or pivot is working.
  • Build a support network rather than trying to carry the startup alone.

02Key sections

Choosing what to ignore
The author highlights the idea that deliberately ignoring tasks is a strategic time-management decision. Applied to products, this means assuming you cannot build everything and focusing on the single most critical piece.
Strategy over tactics in conversion
A recommended article argues that conversion optimization strategy matters more than isolated tactics. The author suggests anyone running A/B tests should read it.
Statistical mistakes in A/B testing
A cited piece explains a common statistical error in running tests. The author notes that testing is only useful if you can tell what results you can trust.
Evaluating a product pivot
A post on pivoting is recommended for its practical guidance on judging whether a product is working. The author notes that the term is overused but that few people know how to evaluate it.
Building a support network
The final pick encourages product builders not to be heroes and to rely on a supportive network, since building products and companies is hard.

03From the post

“Each week, I read dozens of blogs, countless articles from my Twitter stream, and one or two books. In Weekend Reading, I highlight the best, as it applies to developing great products. A Personal Approach To Organizational Time Management by Peter Bregman "choosing what we are going to ignore may”

“choosing what we are going to ignore may well represent the most important, most strategic time-management decision of all.”Teresa Torres · Product Talk

Summary and takeaways written by PM Atlas; quotes are short excerpts. © the original author.