Create a Theme Experiment
Step by step walkthrough for building and launching a theme experiment in Elevate A/B Testing.
Five setup steps, then QA, then launch.
Before you start
Your alternative theme is in Online Store > Themes and unpublished.
It is fully customized across every page type.
Apps that inject into your storefront are configured and working on it.
You have previewed it in Shopify and it looks right.
Step 1: General info and goal
Open Experiments and select Create Experiment on the Theme Experiment card.
Enter an Experiment name. Something you will recognize later, such as "New Dawn theme vs current".
Optionally add a Description and a Hypothesis. Record what you expect and why before you launch.
Optionally attach Assets: designs, briefs, notes.
Choose your goal metric under What do you want to measure?
Revenue per visitor is recommended and is usually right for theme tests. A theme changes the entire shopping experience, so a single narrow metric rarely captures whether it worked.
Select Continue.
Step 2: Set variations
Under Number of Variations, choose how many themes to compare, up to five including your control.
Your published theme is set as the Control automatically. Its name and ID are shown, with a live preview.
For each variation, choose how to supply the theme.
Select
The theme already exists in your library. Pick it from the list.
Duplicate theme in Shopify
You want to test a modified version of your current theme. Elevate opens Shopify so you can duplicate it, make your changes, and come back.
Book a help call
You would like help from the Elevate team setting the experiment up.
The theme picker shows each theme with its creation and last updated dates, so you can tell versions apart. Your live theme is marked Theme In Use. Choose a theme other than that one.
Name each variation something meaningful, such as "Minimalist" rather than "Variation 1".
Use Preview on any card to open that theme, and Edit in Shopify to make changes to it.
Select Continue.
Duplicating a theme
If you chose Duplicate theme in Shopify:
Select Open Shopify Themes.
Duplicate your current theme.
Make your changes to the duplicate.
Save, and leave it unpublished.
Return to Elevate and select the duplicate from the theme picker.
Step 3: Traffic allocation
The default is an even split. Leave it there unless you have a reason not to, because an even split reaches a result fastest.
If you want to limit exposure while you build confidence in a new theme, weight the control higher, for example 70/30. The trade is a longer run.
Isolate traffic for this experiment is optional and off by default. See Traffic Isolation.
Select Continue.
Step 4: Audiences
Every visitor is eligible by default.
Useful narrowing for theme tests:
Mobile only, if the redesign is mostly about mobile layout
New visitors only, for unbiased first impressions from people with no habits on your store
One country, if the redesign targets a specific market
Every filter shrinks the pool and lengthens the run. See Audience Targeting.
Select Continue.
Step 5: Review
Check the summary:
The right theme is assigned to each variation
Traffic split is what you intended
Audience targeting is what you intended, and no narrower
The experiment is created in Preview. It is not live and no visitors are affected.
QA before you launch
Do not launch straight from here. A theme experiment changes every page for every assigned visitor, so QA matters more than for any other type.
Work through Preview and QA a Theme Experiment first.
Launch
When QA passes, select Launch Experiment. The experiment moves to Running.
Watch it closely for the first 24 hours. Theme tests affect every visitor, so a problem you missed in QA shows up quickly and costs more than it would on a single page test.
Do not publish a variation theme while the experiment is running.
Last updated