Run Experiments Before You Write Code

Teresa Torres argues that the Lean Startup loop of Build, Measure, Learn starts in the wrong place, and that teams should instead Learn, Build, Measure. Building first means writing code for features that often fail, which wastes scarce engineering time and adds product complexity that is hard to remove later. A better approach is to surface the assumptions behind an idea and test them with existing data, interviews, observation, or surveys before any code is written. Using a Facebook auto-play video example, she shows how assumptions can be tested cheaply and how they reveal whether an idea is worth building. She concludes that running many small experiments up front, instead of building many features, leads to a leaner and more effective product.

01Key takeaways

• Ask what assumptions must hold true for an idea to work before writing any code.
• Treat assumptions as hypotheses and test them with existing usage data first.
• Set clear success thresholds before analyzing data to reduce bias and focus the analysis.
• Use interviews, observation, surveys, or usability studies to understand why behavior differs across users.
• Recognize that every feature carries long-term maintenance, interface, and learning costs.
• Run many cheap experiments on ideas and build only the few that prove valuable.

02Key sections

The flaw in the feedback loop

Torres agrees with the Build, Measure, Learn loop but says the starting point should be learning, not building. Starting with code means spending scarce engineering effort on experiments that may fail.

The hidden costs of unnecessary features

Features that add no value are hard to remove because teams become attached to them. They add lasting complexity for users, marketing, engineers, and the interface.

Test assumptions before building

Ideas rest on assumptions, which are hypotheses in disguise. Listing and testing them with data or lightweight research shows whether the idea is worth pursuing without writing code.

Investigate the reasons behind the data

When data reveals a split in behavior, qualitative methods such as interviews, observation, surveys, and usability studies can uncover the situations that make a feature harmful for some users.

Slow down to go faster

The upfront experimentation feels slow, but building a weak feature costs far more in maintenance and clutter. Running ten experiments and building the two or three that work is a better use of effort.

03From the post

“The Lean Startup has a flaw. It’s a simple one. It advocates the feedback loop: Build -> Measure -> Learn. I agree wholeheartedly with this loop. The flaw is in where you should start. I prefer: Learn -> Build -> Measure It’s a subtle difference, but it’s an important one. The”

04Frameworks mentioned

Summary and takeaways written by PM Atlas; quotes are short excerpts. © the original author.