Validating product ideas using the RAT

Time, money and resources as we all know are finite. Bummer eh? So when it comes to validating product opportunities, how do we decide which ones are ripe for success and which ones are destined to fail?

“We'll build an MVP!”

While it might seem like the right to do; building a Minimum Viable Product (MVP) can be costly, time consuming and potentially end up being redundant. 

Instead, we should turn to Riskiest Assumption Tests (RAT) as a way of quickly testing the riskiest assumptions we may have about a product, before actually building it.

This is exactly what we did when we were tasked with designing and building a new digital loyalty card for our client - the first of its kind in Europe!

Why Risky Assumption Tests (RAT)?

Risky Assumption Tests have many benefits when it comes to validating product ideas and understanding customer needs.

1. They force you to prioritise. By testing the riskiest assumptions first, you can quickly assess whether they’re a show stopper or whether you can move on to testing the next riskiest assumption.
   For example, you may assume that the best way to reduce loneliness in the elderly is to create a new online chat room. However, if you find that 90% of your users either don’t have the internet or aren’t comfortable using it, then this idea quickly loses its merit.
   
   
2. They’re lean and focused - Risky Assumption Tests are explicit, meaning you only need to build enough to test the assumption. This prevents prototypes or MVPs from inadvertently becoming fully fledged products.
   
   
3. They’re agile - Risky Assumption Tests are a great tool to use in agile product development. They allow you to start small by designing, testing, and validating in the discovery and alpha phases, before building the product in the Beta phase. They don’t demand huge amounts of resources, allowing you to quickly pivot if needed.
   
   
4. They put the focus on learning - By quickly testing your assumptions, the focus is on learning, rather than on building. These learnings help build a shared understanding among stakeholders leading to more informed decision making, ensuring you approach your build from a position of knowledge.

How to run Risky Assumption Tests?

Ok, so I’m sold on Risky Assumption Tests, but how do I go about running one? 

Step 1: Declare

The first step is to declare your assumptions. This should be a group exercise and involve everyone who has a stake in the product or can provide specific subject matter expertise. Give them advanced notice so they can come prepared with any materials they need such as usability reports, analytics, info on prior projects etc.

Have everyone write their assumptions on a post-it note and put them on a wall. An example of an assumption we wanted to test for our client’s new digital loyalty card was:

"We believe customers want a digital loyalty card because they keep forgetting their physical card"

If you are struggling to come up with assumptions you can use an assumptions worksheet as a template to get you going.

You can also create a problem statement, which should detail the goals, problem to be addressed and potential improvements you are seeking. This statement will be filled with assumptions you can test.

Step 2: Prioritise

Now you have all your assumptions listed out on a wall it’s time to prioritise them! Remember, we want to test the riskiest assumptions first. 

Before doing this you may need to do an affinity mapping exercise to group your assumptions into themes if participants have come up with similar assumptions.

Next plot these assumptions on the prioritisation matrix below. We generally want to prioritise them in terms of riskiness (i.e their importance to the product's success) and the level of unknown (how confident we are that the assumption is correct).

Step 3: Hypothesise

Next, we want to turn these assumptions into testable hypothesis statements. Using the example assumption above our hypothesis statement might be:

We believe customers want a digital loyalty card because they keep forgetting their physical cards

We’ll know we’re right when we see customers trying to get a digital loyalty card from our website

Some hypotheses, such as the one above, are too big to test and need to be broken down into smaller sub-hypotheses, for example:

Sub-hypothesis one

• We believe that creating a digital loyalty card
• For existing loyalty customers
• Will mean they add the digital card to their phones
• We’ll know we’re right when we see customers trying to get a digital loyalty card from our website

Sub-hypothesis two

• We believe that creating a digital loyalty card
• For existing loyalty customers
• Will mean they forget their cards less
• We’ll know we’re right when we see forgotten card codes requests reduce during purchases

Step 4: Test

Once you have your hypotheses, it's time to devise a way to test them. You should keep these tests as lean as possible and do just enough to get the feedback required to prove or disprove them.

Using sub-hypothesis 1 above as an example. We may decide that the easiest way to test this is to add a link on our website, which allows customers to get a digital loyalty card. This could be something as simple as an HTML link which takes the user to a holding page. 

Another way we may want to test this is to run a poll on our website asking customers whether or not they would download a digital loyalty card to their phone.

The way you test it is up to you, but I find the best way is to start from the data/feedback you want, and then work backwards creating the test from there. Bear in mind you may also be able to test multiple hypotheses at once with one test.

Step 5: Analyse

Once you have run your test it's time to analyse the results. Here you must decide if the hypothesis has been proved or disproved. It’s at this point you must also decide if you want to test the next hypothesis or if in fact, it’s not worth going any further.

For example, if we disproved sub-hypothesis one above, we’d need to ask ourselves whether it’s worth testing sub-hypothesis two. Because if customers aren’t going to add the loyalty card to their phone, it’s going to be no use to them when they forget their physical card.

Step 6: Repeat

The process above is great because it allows you to continually feed new assumptions into it and test them until you reach a point where you are comfortable you’ve tested enough assumptions to start building something more concrete.

RAT before MVP

The next time you’re in a meeting and someone recommends creating an MVP to test a product idea, take a step back and think of the RAT. It could save you a lot of pain.

If you’d like to find out more about how Risky Assumption Tests can help you, get in touch.

References

“Lean UX” - Jeff Gothelf