Planning experiments
Constructing a hypothesis
Experimentation is a powerful tool to gain insight into how your customers behave. Insights enable better, more frequent iteration and integration improvements.
The first thing to start with is a clear hypothesis which clarifies the problem you are trying to solve, focuses your experiment and provides a measure of success. Look for a problem with your current integration and think about something that you could change to improve it. For example, has your offer impression volume decreased, or is customer engagement lower than you would like?
Testing a hypothesis in a controlled way allows you to ask a pointed question about how a different design or experience might change your customer’s behaviour, and collect valid and reliable data to answer it. Make sure your hypothesis clearly articulates the change you are making to the page (e.g., the colour of the positive response changed from button from blue to red) and identifies the metric you want to affect (e.g., revenue). When measuring the success or failure of your experiment you should look at your primary metric to ultimately decide whether your experiment proves or disproves the hypothesis.
Once you have a hypothesis, you’re ready to build your experiment.