Experimentation using for instance lean startup can help you learn about your customers and find out which features and product would be valuable. The value however comes from building products and actually delivering them to customers. You need to find ways to balance between experimentation and delivery.
In the Huffington post article 6 myths about lean startups Keith Hanson explains why the myth “the Lean Startup model encourages features vs. whole products” is not true. He describes how the build-measure- learn loop from lean startup with hypotheses and experiments supports to deliver whole products:
What the Lean Startup methodology teaches us is that the dichotomy between "feature by feature" and "my product is a massive behemoth of everything the customer could possibly want" is a false one. Rather, it is about allowing the metrics that matter and the business goals associated with those metrics to drive the priority of development tasks. Customer development and build-measure-learn feedback loops are what determine how a product is developed in the Lean Startup methodology. Not the false dichotomy of features and complete products.
(…) I've found that entrepreneurs generally think about these questions and answers intuitively anyway, and without any guidance, end up incrementally improving their products and value propositions. Lean helps entrepreneurs refine those hunches, create actionable hypotheses to test, and record learning for future decisions, all the while targeting key success metrics for the business.
Roman Pichler wrote the blog post get your focus right: learning and execution in scrum in which he explained the need to experiment and learn:
When we start developing a new product or new features, there are usually more unkowns than knowns, more things we don’t know than we know: We may not be clear on the user interaction, the user interface design, the product’s functionality, or the architecture and technology required to build the product. Our greatest challenge is therefore to deal with the uncertainty present, and the associated risks.
Gil Zilberfeld wrote about applying lean startup in I have an idea. He defined a form with two parts which can be used to test ideas with experiments:
In the Hypothesis part, we describe the feature from a customer point of view:
- What is the idea?
- Whom does it help?
- What does it solve?
In the Experiment part, we describe how we can learn if our hypothesis has merit:
- What are we going to do?
- Who are we going to show it to?
- How are we going to measure it?
- How much is going to cost us?
- How will we know if it succeeded?
During development you have to move your focus from experimenting to delivering, as Roman Pichler explains:
As you acquire more knowledge, your focus should gradually shift from resolving uncertainty towards execution: building a product that is ready for general release. Rather than primarily testing ideas, you should now start completing features and incrementally adding new ones.
Tim Ottinger wrote the blog post what is all this about failure in which he explores experiments to support decision making in product development:
Any problem that is interesting will involve experimentation and learning. Those problems cycle on steps like these:
- Try (do your best)
- Fail (intentionally try the stuff you're not sure about)
- Investigate (find out what you didn't know)
- Learn (incorporate new knowledge)
- Exercise options: pivot, persevere, abandon, expand
He describes four options that you can exercise based on the learnings from doing experiments:
Pivot is a kind of success because we choose a different path, one that has greater promise. (…)
Abandon means that we have succeeded in saving money and time because we learn which problem is not actually worth solving, or the solutions to the problem are too expensive to bother investing in. (…)
Persevere is what we do when we learn that our current direction has promise, or that nothing else looks more promising. (…)
Expand is what happens when we know we have a great idea, and that we can leverage it in ways we hadn't expected.
Experimentation provides insight in the options like continuing and start delivering products or changing direction which may require additional experiments. However you will have to decide fast enough to take advantage from the experiments:
If we don't fail early enough, then our options begin to expire. We may no longer have the choice of expanding in a promising direction, shutting down expensive time-wasting ideas, etc. In short, we have no choice but to hunker down, work harder, and hope that it comes out okay.
The authors Jeff Scheinrock & Matt Richter-Sand from agile startup explain in stop experimenting, start building that that you have to balance between testing ideas and building a business:
Inevitably, there comes a time when you need to either start building the business or nix the idea entirely. It’s easy to convince yourself that you need to do a few more tests before committing. Like it or not, you’re running out of runway and you need to take action one way or the other. Don’t let analysis cause paralysis.
You will never be 100 percent certain of anything in a startup. The key is to balance action with experimentation in a way that keeps your business moving forward.
