Test your assumptions before building your product
Now that I am teaching and advising on product management, I get to coach PMs and founders on the craft of building products.
Here is a common dialogue I go through.
PM: What should be the MVP?
Me: Depends. What is your hypothesis to test?
PM: What should be the hypothesis?
Me: Depends. What is the biggest assumption or risk on which you are banking your product on?
There is a lot of talk on MVP and its definition. When should you build it? What it is ? What is the difference between MVP and prototype and version 1? I will tackle this separately.
Don’t think MVP
Instead of asking what my MVP should be, first ask what is your biggest assumption. When you are starting out as a new startup or even building a new product in an existing company, you are making assumptions.
Your assumptions are roughly in this order :
- Is there a customer need?
- Is this a one off need for a few people or a need for many?
- Is this a high priority need?
- Will they actually use it? Will users change habits?
- Will it actually work and provide value?
- Can we build it ? Profitably ?
- Can we scale GTM?
These are the starting assumptions at least for new products. For existing products, there are other risks like retention risk, competitive risk, economic risk etc.
Bringing a new product to market carries risks throughout its life cycle. As a founder or a product manager, you need to be aware of these risks all the time.
When you start a new product, the biggest risk is if there is even a need. As you progress in your products journey, your risk profile changes.
At any given point of time there is one assumption that is is the riskiest. And that’s the risk you want to test for. Once you test for that, then move to the next riskiest item.
For this article, let’s focus in the initial set of risks for a new product. The goal is to test your assumption before expending any time or resources on building a product
So instead of asking what your MVP should be, ask what’s the biggest assumption or risk is at this time. Then create a hypothesis based on that assumption. And then design the cheapest experiment. There are many ways to cheaply experiment and prove your hypothesis. You don’t have to build something at this stage.
This is an example of how Zappos did it.
Assumption or Risk: Will people buy shoes online? Tony Hsieh wanted to use the internet to sell shoes online. Amazon had already started the online model with books. But shoes? He was not sure and wanted to confirm that assumption.
Hypothesis : X% of people who visit our site will buy shoes online. ( I don’t recall the exact percentage he had in mind.)
Experiment: Create a web site with pictures of shoes people can buy. Tony would go to a shoe store and take pictures of shoes from his camera. Then upload them on his site. When someone ordered, he would get an email, go back to the store to buy the shoe and then ship it.
Did he invest in supply chain, pricing optimization, SEO, UX?
NO. All that are super critical and would come later. But what’s the point of investing in all that when no one will actually buy a shoe online. That’s a mistake Webvan did around the same time and spent almost a billion dollars building the full product for online groceries.
Let’s go through the key risks for your new product and what you can do to validate your assumptions.
Need Risk : Is there a need?
One of the classic mistakes I have seen from many startups is they think they have built a solution based on their own experiences. If I had this problem, others have it too.
You think customers might have a pain point because you have experienced it but when you speak to customers, it may turn out they don’t consider this a pain point. This is the holy grail of product management and the most risky of product assumptions that needs to be uncovered during discovery.
Talk to your target customers. In B2B, talk to at least 10-15 potential prospects in the segment you are targeting. Ask them about the pain point. Confirm that they actually have a problem.
Sometime the problem only exist in our heads. We need to ensure that it is also in our prospects head. And talking to customers is a best way to validate your assumption about a need.
The point of talking to customers is to reduce this risk. If this risk holds true, then your product or business will not go anywhere. If you are in B2B, working on a new product in a new space, you have no option but to talk to customers and confirm the pain point.
Priority Risk: Customer has pain point, but its not top of mind
In B2B, business customers have a lot of pain points. They are always battling business challenges related to marketing, sales, operations, production and so on. They simply cannot work on all the challenges at the same time. So they prioritize.
How? They prioritize based on what is important to them as a business at that time. And when you are selling them a product, it has to be a pain point that is a high priority for them.
Otherwise they will simply defer or not even bother. This is something you have to uncover as part of your discovery process.
You can do some experiments to determine how they are solving the problem today. Let’s say you are helping sales team with AI recommendations that helps improve pipeline quality and conversions. Create a landing page with the right messages and see how many prospects sign up. You can offer them a white paper or a check list of things that helps improve pipeline. Create a lead magnet that helps you validate that this is something a customer wants to solve today.
If no one signs up to your free offer, then you know it’s not a priority.
Small Market Risk: Customer has pain point, but not many have this problem
There are two variants within this risk.
First, there is a pain point that you uncovered in a couple of discovery sessions, but not many customers resonate with that. Find out why this pain point is not applicable to them. What other pain points do they need to solve?
Second, the market is not too big. Imagine you want to build a CRM for Llama farmers. You spoke to a couple of them and they both mentioned a need for s system to manage their customers. But how many llama farmers are there.
So this is a market size risk.
You have uncovered a pain point in a couple of discovery sessions. But how do you ensure that this is not a one off need. This pain point should be more prevalent across a large set of customers. You need to speak to many many customers to ascertain a pattern. If many customers report the same pain point, then you have a market to go after.
Your initial discovery will give you clues on how prevalent the pain point is.
Usage Risk : Will users actually use it?
In the book “The Right It”, Albert Savoia write about the Palm Pilot. In the mid 1990s, there was a company that create the first personal digital assistant (PDA), way before iPhones and BlackBerry. You could look up your appointment calendar, find the phone number of someone you want to call and take notes.
This was a physical device which meant spending a lot of R&D money to even develop a prototype. What if after building this expensive prototype, few people use it. Changing habits is a really hard thing to do. This is what the founder did.
He create a small wooden box with the rough dimensions of his planned device. In the box he placed sheets of paper. On the paper, he noted down his daily appointments for the week.
His assumption was will he ever take out the box from his pocket to check his appointment. His hypothesis was when I need to know about my next appointment, I will check against my “device”. After a week, he concluded that majority of the time he used the device. He tested this with a few colleagues as well. Only when they were sure that people will rely on this device, they proceeded to build it. Of course, Palm Pilot was a massive success until new tech aka smart phone came along.
You may have the best technology and you actually might even have evidence that this is a big user pain point to solve, but the risk is if people will actually use it. The Zappos MVP experiment is a good example as well.
Look around your home and see how many product you have bought that you have hardly used. Look into your kitchen for gadgets or utensils you hardly used.
That’s a risk. Because if people don’t use your product, they will not renew, they will not recommend and your product will languish. So it is critical to test your hypothesis that at least some people will actually use your product when they need it.
If you are building a SAAS product, how will you test for this hypothesis?
This is where you can get creative. I was recently advising a product team building a mental health app. I asked that when people are in a mental crisis, will they actually remember to use your app and get help. They assumed they would.
I recommended they create a simple google form and ask some questions. Make the link to the form easily available on your site. Now check how many people will actually come to the form. Now they did not tell users it is a form. They just said come to this page for help. In a week, 40% of their audience actually visited the form and answered the 2 questions. That proved that when they need it, they will use it.
Your hypothesis should be something like “When a user needs to do a Job X, at least 40% of my target will actually use it”
Create the lowest cost experiment to test this hypothesis. With no code, landing pages, clickable screens, you have a lot of options. Proceed to build your product once your hypothesis is validated. Be creative.
Here is a brilliant example of how one founder tested an idea of a product in Ikea.
Note: Usage risk above is not the same as usability risk. Usage risk happens if the user does not even use the product when it is needed. It’s not top of mind or they forgot about the product.
Usability pertains to ease of use and is covered under solution risk below.
Solution Risk: Solution does not solve or is a hassle to use
At this point, you have validated that there is a real customer need and they will use your product when needed. Now you can proceed to build the product. But there is another risk looming.
What if it does not work?
What if it is too much hassle?
At this stage, you will likely need a minimal product. In this minimum product, only build that core feature that you plan to solve for the user which will be your main differentiation. e.g. You want to build a sales recommendation tool using AI for sales reps to find the best opportunities in their pipeline. Create a bare minimum feature using AI that provides recommendations. It can be a simple one page app. Now test if the recommendations are accurate and do they work most of the time.
The risk here is that customers do not find the product valuable. By creating a low fidelity prototype, you validate that the product will actually create the value for your customer. It will also help you determine how to reduce friction in usage so that it is not a hassle to use.
e.g. maybe this tool for sales rep should be integrated with their CRM, otherwise it will be too much of a hassle. What other integrations might be needed?
Remember, at this stage whether this is a priority for them is not an issue. You already tested that before. The risk is they will use it but not find enough value.
The key thing to remember for your prototype is that you only need to test for the real core value proposition. You don’t need to test single sign on, audit trails, fancy dashboard or SOC 2 compliance. These bells and whistles will come in later.
Conclusion:
Don’t think of MVP from a product perspective. The P in the MVP is a misnomer. It was coined back in 2008 and the name has stuck.
It should really be called MVE…an experiment. As you can see from some of my examples above, most are actually not products. They are experiments to test your hypothesis.
Find the riskiest of assumptions at any point of time and test for that before you move to the next assumption. This is a much better way to validate than creating an MVP. What is the cheapest way your can test your assumption, without having to build the product.
Follow this process.
Assumption : What is my assumption or risk that I need to validate or invalidate? Hypothesis: How will I know if the assumption is valid. Create a statement like this ” of the X number of people who try, Y% will do Z”
Experiment: Design the cheapest and fastest way to experiment your hypothesis.
Do you have to test every risk?
If you are building an improved version of a product that is better than the incumbents, then you don’t need to test the need or priority risk. Customers are already using something today which means they have a need and they will use your product. Your risk is if your product is actually better than the incumbents. That’s where you can start.
If you are creating something new, then you don’t know if there is a need, let alone if users will use it. So you start your experimentation from the beginning.
You can download my template here to identify the types of risks a new product faces.
