Shipping Testing Overview
What a Shipping Experiment tests, the Shopify carrier-calculated rates requirement, how rates are delivered at checkout, and how shipping revenue is counted differently from other experiment types.
A Shipping Experiment tests shipping rates, free shipping thresholds, and delivery options against each other. Different groups of visitors see different shipping options at checkout, and Elevate A/B Testing reports which set produced more revenue and more completed orders.
Shipping is one of the few levers that changes both sides of the equation at once. A lower threshold can lift conversion and cut margin at the same time, and the only way to know the net effect is to measure it.
Requirements
Shopify plan
Your store needs access to Shopify's third-party carrier-calculated shipping rates. That comes with Shopify Advanced and Shopify Plus, with annual billing on other plans, or as a paid add-on. This is a Shopify platform requirement, not an Elevate A/B Testing one.
Carrier rate in your zones
Elevate A/B Testing adds its carrier rate to the zones you selected when the experiment launches. You can also add it yourself in Shopify beforehand.
Theme extension
Only needed if you also change on-page shipping messaging per variation. The rates themselves do not use it.
Variations
Up to five, including the control.
If you are on a monthly plan without carrier-calculated rates, switching to annual billing or contacting Shopify Support to enable the feature are the two routes to running this experiment type.
What you can test
Free shipping thresholds. Whether free shipping at one cart total beats free shipping at another.
Flat rate pricing. Whether a cheaper rate wins enough extra orders to pay for itself.
Free shipping against paid shipping. Whether removing the charge entirely grows total revenue or only shifts it out of your margin.
Delivery speed options. Whether a faster estimate, or a different mix of speeds, converts better.
Conditional rates. Different rates by cart total or by order weight.
What you can configure
Each variation holds one or more rates, and each rate supports:
Shipping Rate Name
The shipping option name the customer sees at checkout
Rate Description
Optional text shown alongside the rate at checkout
Shipping Price
The shipping cost, in your store's currency
Delivery Time
Optional minimum and maximum delivery estimate shown to the customer
Enable Conditional Shipping
Makes the rate apply only within a range of cart price or order weight
Because a variation can hold several rates, you can test a whole shipping menu rather than a single price. One variation might offer a standard and an express option, while another offers free standard and a cheaper express.
Free shipping thresholds are built with conditional shipping rather than a dedicated threshold field. A free rate that only applies above a certain cart total is a conditional rate. Variations group their rates by kind, so flat rates and conditional rates appear under separate headings.
Shipping profiles and zones
A Shipping Experiment runs against your existing Shopify shipping configuration. During setup you select the zones the experiment runs in, then choose which of the existing rates in those zones the experiment replaces.
Your profiles, zones, rates, and currency all come from your own Shopify setup, so what you see at each step is specific to your store.
One experiment runs in one zone. Testing a second zone means running a second experiment, which also keeps the two results readable separately.
Keeping a test contained is therefore automatic. Testing domestic rates leaves every other zone untouched.
How it works
Elevate A/B Testing registers with Shopify's Third-Party Carrier Service API and returns rates on request.
You configure a set of rates for each variation.
At launch, Elevate A/B Testing adds its carrier rate to the zones you selected and hides the rates being tested.
A visitor reaches checkout and Shopify requests shipping rates from Elevate A/B Testing.
Elevate A/B Testing checks which variation that visitor belongs to and returns that variation's rates.
The visitor sees only the shipping options for their variation, and the order is attributed to it.
Your original rates are restored automatically when the experiment ends or is paused. You do not have to put them back yourself.
Assignment is stored in a cookie, so a returning visitor sees the same rates they saw before. From the customer's side the checkout looks entirely normal.
What a Shipping Experiment cannot do
It cannot be limited to specific products or collections. Rates apply store-wide within the zones you enable. Every product that ships through those zones uses the test rates.
It cannot run without carrier-calculated rates. There is no workaround, because the capability comes from Shopify.
It cannot leave your existing rates visible. For the rates being tested, your own rates are hidden while the experiment runs, because two competing sets of options at checkout would confuse customers and corrupt the result.
How shipping revenue is counted
Every other experiment type excludes shipping fees from revenue. Shipping Experiments include them, in Revenue, Revenue per visitor, and Average order value alike.
The reason is that the shipping fee is the thing being tested. Excluding it would hide the financial effect of the change you made, and a variation that charged more for shipping would look identical to one that charged nothing.
Expect your revenue figures on this experiment type to sit higher than figures from your other experiments. They are counting something extra on purpose. See Reports Overview for what each metric means and Statistical Significance for when to trust the result.
Cart line item properties
Shipping Experiments attach data to cart line items so the right rates and attribution follow the order. Shopify hides properties whose names begin with an underscore, so on most stores customers never see them. Some custom themes and apps ignore that convention and display them anyway.
If experiment data is showing up in your cart, order confirmation emails, or packing slips, see Hide Cart Item Properties.
Where to go next
Create a Shipping Experiment: profile and zone selection, rate configuration, and launch.
Preview and QA Shipping Experiments: the pre-launch checklist, including the scenarios worth testing at checkout.
Troubleshooting Shipping Experiments: rates not appearing, wrong rates showing, and messaging that does not match.
Audience Targeting: limit the experiment to specific devices, visitor types, countries, sources, or UTM parameters.
Traffic Isolation: reserve traffic so those visitors join only this experiment.
Ending an Experiment: how to end a Shipping Experiment and what happens to your rates afterwards.
Last updated