By Marty Cagan · svpg.com · LinkedIn
Cagan distinguishes live-data prototypes, which are real code sent live traffic (usually in A/B tests) to prove an idea works, from production software that the business can run on. Production demands complete critical use cases, test automation, scale and performance engineering, SEO, internationalization, code reviews, and sales and support training. Most of that is unnecessary for a live-data prototype and works against minimum viable product goals. Teams that confuse the two tend to over-engineer experiments or, worse, let the company sell something prematurely. A live-data prototype still needs to handle real traffic and have analytics, typically 20-50% of production effort, and once proven it must be productized.
01Key takeaways
- Be explicit with the team whether a backlog item is a live-data prototype or production-quality work.
- Avoid production-grade engineering like full automation or internationalization during discovery experiments.
- Label live experiments internally so sales and marketing do not sell or promote unvalidated features.
- Ensure a live-data prototype handles real traffic reliably and includes analytics to measure its success.
- Performance engineering may be necessary when slowness would distort the test result, such as in e-commerce.
- Plan and budget for productizing a validated prototype into production software.
02Key sections
- Defining production software
- Production software is software the business can rely on, with critical use cases tested, automation, scalability, localization, and trained go-to-market teams. These requirements explain why going from working code to a sustainable product takes real time.
- Why prototypes should avoid production rigor
- Product discovery seeks the fastest, cheapest validation of a hypothesis, so most production requirements add waste to a live-data prototype. Over-engineering slows iteration.
- Risks of confusing the two
- Teams may over-engineer experiments without clear labeling, and the company may mistakenly sell or advertise an experiment, creating customer problems. Clear communication about status is essential.
- Minimum bar for a live-data prototype
- A prototype must handle live traffic, provide analytics to measure success, and avoid harming customers, costing roughly 20-50% of production work.
- Special case and productization
- Performance can be essential to fairly testing an idea, especially in e-commerce, so some performance engineering may be needed. A successful prototype must later be productized, which should be planned for.
03From the post
“A partnership dedicated to teaching best practices to product teams and product leaders”
04Frameworks mentioned
Summary and takeaways written by PM Atlas; quotes are short excerpts. © the original author.