I would like you to do a small quiz built by a person I admire a lot: Dan Ariely, the behavioural scientist. The quiz was designed to assess one’s ability to make government policy and take corporate decisions. But I believe it is at least as appropriate to assess your ability to be an entrepreneur.
Here’s the deal:
No ad to show here.
Take this Dan Ariely quiz. When you finish, read Dan’s feedback. Return to this page and get some insight on its relevance to entrepreneurs and startups.
How did you do? Did you get the right answer (ie an increasing sequence of numbers) or did you get tricked by your subconscious confirmation bias. Don’t worry if you were in the latter category as that puts you in the majority.
So other than the fun of a quiz – what’s the point?
Anyone starting a new business, be it anything from a coffee bar to a tech startup, should really be using the priniciples of the lean startup process.
If you have not heard of it, at the most fundamental level, the lean startup suggests that conceiving and growing a new business should be a series of experiments designed to help you uncover a sustainable business model.
Two of the key elements in using the lean startup process are setting the experiments to test the hypothesis and the analysis of the results. This is where confirmation bias could trip you up.
Here are some examples where confirmation bias might lead you astray:
1. At an early stage of your venture, you test with some family and friends if they think the idea is good. Nine out of 10 said yes and so you rush on into the development process.
Potential error: As described in our recent myths article, friends and family tend not to tell you the truth in this situation if they do not think it is a good idea. (Plus you probably would write off the one out of 10 who are negative as just being “their normal cynical self”!)
Better test: Ask if they would invest money in your venture – and go so far as to ask for it. You can always back off later. They may still not give you a straight answer, but the number of people who say “yes” to investing in you is a much stronger test.
2. You conduct an online survey to test if your market really does face the problem you are trying to solve. Seventy-five percent say that it is a problem and so you conclude your hypothesis is right
Potential errors: Did you test if it is really a serious problem? Do they have a current solution or workaround? Was your sample truly representative of your market (size; segment)? Was the problem statement at too high a level to really uncover the issue? Did you structure the question in a leading manner?
Better test: Use open-ended questions in a less structured, face to face interview conducted by YOU. You will get far more granular insights. We are not averse to online surveys, but they should be used with caution as confirmation bias can make it easy to misread the results
3. You test the level of virality of your product with early users. Three out of 10 recommend it to others. You conclude that virality will be your engine of growth for new users.
Potential errors: Your users may simply be a small segment of early adopters who are closely affiliated and like to show that they are in the vanguard of new ideas. When you start reaching your mainstream market, that same attitude and/or close affiliation may not be present and far fewer recommendations are made.
Better test: Test across a broader cross-section of markets. Cynically analyse the different results and try and get to an understanding of why referrals are made. Look at the non-referrals and see if you can see why referrals were not made.
Confirmation bias largely lies in our subconscious and so it is not always an easy thing to avoid. However, if when setting hypotheses, developing experiments and analysing results you actively seek out possible areas of confirmation bias, you are far less likely to fall into the traps. Things to consider are:
- Look out for tests that might produce a false positive (e.g. friends and family in 1 above).
- Use co-founders, advisers and other external parties to play the devil’s advocate and challenge you.
- Assume a null hypothesis (i.e. that your hypothesis is wrong) and test for that; as well as simply testing to see if your hypothesis is right.
- As much as possible work with real customers in live situations rather than remotely.
- Use formal techniques like Validation Boards and group reviews of results to test your hypotheses; and encourage an atmosphere of criticism and honesty.
- This is far easier said than done; particularly as entrepreneurs tend to have a positive and optimistic view of the world and their ventures. However this simply makes it even more important to try and maintain a meta level of thought to catch yourself falling into these traps.
This article by Simon Gifford originally appeared on Mashauri.com and is republished with permission.
