How to Create a Shipping Experiment

This guide will walk you through creating a shipping test using Elevate A/B Testing to optimize your shipping strategy.

Step 1: Create a new Experiment

To get started, go to the home page and click the ‘+ New Experiment’ button in the top right corner. This will open a modal where you can select the type of test to run. Click on the Shipping Experiment card to begin setting up your test.

Step 2: Enter Test Details

• Name Your Experiment: Provide a meaningful name for your experiment to easily track its progress.

• Add Description and Hypothesis: Although optional, including a detailed description and hypothesis can enrich your experiment, outlining the purpose behind the test and what you anticipate discovering.

Step 3: Choose Your Shipping Test Type

In this step, you’ll select the type of shipping test you want to run. You can choose between three types of shipping experiments, depending on your goals:

• Free Shipping Threshold Test how setting different free shipping thresholds affects your conversion rates and average order value.

• Flat Rate Shipping Experiment with different flat shipping rates to determine how consistent pricing affects customer behavior.

• Advanced Shipping Experiment Run advanced tests by combining multiple shipping factors, such as delivery speed, shipping zones, or carrier options.

Step 4: Select Shipping Profiles and Zones

After choosing your shipping test type, select the shipping profiles and zones where you want the experiment to apply. Shipping profiles represent the shipping options in your store, and zones define the geographical areas where those rates will be applied (e.g., US, Europe). Be mindful of the profiles and zones you select, as they will impact what shipping rates your customers see during the test.

Step 5: Select Shipping Rates to Test

Once your profiles and zones are selected, pick the specific shipping rates you want to test. For example, you might test different free shipping thresholds, flat rates, or a mix of rates based on weight or delivery speed. The selected rates will be hidden during the test and replaced with the experiment’s rates.

Step 6: Set Up Test Variations

Create between 2-5 variations for your test. Each variation will represent a different set of shipping rates or thresholds you’re testing. For instance, one variation might offer free shipping at a $50 threshold, while another offers it at $75. The more variations you have, the longer it may take to reach statistically significant results.

Step 7: Allocate Traffic Percentages and Set a Goal

Distribute traffic among your test variations by allocating a percentage of visitors to each one. For example, you could allocate 50% of traffic to one variation and split the remaining 50% among other variations.

Next, establish a primary goal for the experiment, such as increasing conversion rates or average order value. This goal helps Elevate determine statistical significance once the test is complete. Don’t worry—Elevate tracks all your test data, so you’ll be able to view and analyze everything. The goal simply helps measure the success of your experiment.

Step 8: Add Audience Targeting (Optional)

You can optionally add audience targeting to refine who sees your experiment. Target based on:

• Devices: Choose between mobile, desktop, or both.

• Visitor Types: Distinguish between new and returning visitors.

• Traffic Sources: Filter traffic based on specific marketing channels or UTM parameters.

• Locations: Target specific countries or regions.

These filters allow you to run more precise tests and get targeted insights from different customer segments.

Step 9: Review and Launch

Once you’ve set up everything, review the entire experiment to ensure all the details look correct. If everything looks good, click Create Experiment at the bottom of the page. Your shipping test is now ready to go live, and Elevate will begin tracking results as soon as traffic starts flowing to your variations.

Step 10: Conduct Quality Assurance (QA)

• Review and Test: Before going live, conduct a thorough QA of your test setup to ensure that all elements function as intended and that the test variations accurately reflect the changes you're testing.

By methodically following these steps, you're well-equipped to execute a page experiment using Elevate A/B Testing. This approach not only facilitates the exploration of different page designs and functionalities but also provides valuable insights to refine user experience and engagement on your Shopify store.