Last updated Oct 24, 2023

Build an experiment

To build a creative experiment, follow these steps:

1. Log in to One Platform, and click on your account on the Home page.
2. From the left navigation menu, go to Campaigns > Experiments.
3. Click Add experiment.
4. Enter an experiment name.
5. Select Creative Multivariate Experiment.
6. Click Continue.

Experiment details

7. Enter a hypothesis. A good hypothesis clarifies the problem you are trying to solve, focuses your experiment, and gives a measure of success. Make sure your hypothesis clearly articulates the change you are making (e.g., changing the copy of the creative) and identifies your main success metric (e.g., conversions per referral).
8. Select a primary metric (and optional secondary metrics). When measuring the success or failure of your experiment, the primary metric determines whether your experiment proves or disproves the hypothesis.
9. Select a Minimum uplift.
   1. Minimum uplift is the smallest relative percentage change you want to detect in the primary success metric. Rokt uses this value to estimate the expected duration of the test, recommend actions, and ensure you collect enough data to measure your desired uplift.
10. Click Next.

Design variants

The next step is to define the experiment variations that will validate or invalidate your hypothesis. It’s important to ensure there is a baseline in place (known as a control) that will act as a point of comparison for any results you receive from your experiment. Below are some rules which guide best practice when designing your variants:

1. Only test one element at a time so that you understand which element triggered the results you are seeing. We don’t recommend changing both the call to action and the landing page simultaneously.
2. Set up control and treatment. Retain a version of your original creative in the experiment; this is your control. The new variations you are testing are called treatments. They are challenger variants you’re testing against your control.

To get started:

1. Select a campaign and creative from the dropdowns. This is an existing creative which will be the baseline for your experiment. Once selected, you can see the audience details for the creative.
2. Click Next.
3. Click New Variant + to define your creative variant.
4. Make changes to the creative variant. Keep in mind the recommendation above to only change one element at a time. For example, change one of CTA, landing page, copy, logo, or promotion value.
5. You can create up to five variants to test against your control.
6. When you finish adding variants, click Next.

Traffic allocation & variant distribution

The traffic allocation step is used to define how many of your customers you wish to include in your experiment.

Enter a numeric value, or use the slider to specify the percentage of all customers viewing the offer to include in your experiment cohort. This is a mechanism to control exposure to your test variants and reduce the risk of harming your overall campaign performance. It is important here to consider how long it will take to generate meaningful results, which can differ based on the scale of your campaign, and how long it takes your typical customer to convert after seeing an offer.

Starting your experiment

When you’re ready for customers to see the experiment, click Start experiment. When the status field says Live, your experiment is running. Most experiments go live within five minutes. You can view all configured experiments on the Experiment Overview page.

How long should an experiment run?

We recommend keeping an experiment running for at least two weeks to ensure you collect enough data. Do not end your experiment until at least one variant has a 95% probability to beat baseline. It could take up to 30 days to gain enough data to make a confident decision about the outcome.

Ending your experiment

If you’ve reached one of the 3 scenarios outlined above, you may consider ending your experiment and applying your learnings by promoting one of the experimental creative variants.

To end your experiment:

1. Go to the Experiment Details page in One Platform.
2. Click End experiment.
3. Confirm you want to end the experiment.

note

Once ended, experiments cannot be returned live

Promote a variant from the experiment:

1. Select Publish variants from the Experiment Details page of an ended experiment, or click on Set Live in the table.
2. Select the variant(s) you would like to promote out of the experiment.
3. Click Publish Now

Congratulations! Your experimental variant is now live. You can select one or multiple variants to promote to your page from the experiment. If you no longer want customers to see the original, make sure you disable the original creative in Campaigns Overview (Campaigns > Overview).