Theme Experiment Troubleshooting and FAQs
Fixes for common theme experiment problems in Elevate A/B Testing, plus answers to the questions merchants ask most.
Troubleshooting
I keep getting redirected to the published theme when previewing
Add eabUserPreview=true to your preview URL. This is the most common issue during theme development.
https://yourstore.myshopify.com/?preview_theme_id=1234567890&eabUserPreview=trueMy team cannot preview the variation theme
Share the preview link with eabUserPreview=true appended. Without it they may be redirected.
I am still being redirected after the experiment ended
Your browser is holding an assignment from the experiment. Clear your cookies for your store domain, or open the store in an incognito window.
Theme editor changes are not showing up
Hard refresh the page, Cmd + Shift + R on Mac or Ctrl + Shift + R on Windows. Also confirm you are editing the theme you think you are: the variation, not the control.
Everyone sees the control theme
The experiment is not running
Check its status in Elevate and launch it
A variation theme was published
Only the control may be published. Unpublish the variation.
You are excluded
You may be holding an eabUserPreview cookie, or an earlier assignment. Use an incognito window.
Audience targeting excludes everyone
Open the experiment and review the filters
The variation theme is being served but records no data
Stop the experiment and contact support. Visitors are seeing the theme without being counted, which biases your result.
Check this during QA rather than after launch.
An app is missing or broken on the variation theme
Apps do not carry across between themes. Configure the app on the variation theme in Shopify, the same way you did on your published theme.
If it is configured and still not appearing, contact the app's support: the app needs to work on an unpublished theme for a theme test to be valid.
The published theme changed mid experiment
If you edited or republished your control theme during the run, the experiment compared two moving targets. End it and start again once both themes are stable.
Results look wrong
Fewer visitors than expected
Audience filters, or traffic isolation on another experiment
Third party analytics disagree
Some tools only fire on the published theme and under report the variation
The variation lost badly
Check content parity. A missing section, banner, or app is a common cause
One or two orders swing the result
Normal early in a run. Wait for more data.
FAQs
Does theme testing work on my Shopify plan?
Yes, on all Shopify plans.
Do visitors know they are seeing an unpublished theme?
No. The store looks and behaves normally, and cart and checkout work as usual.
Will a visitor see two different themes on different pages?
No. Assignment is sticky. A visitor stays on their assigned theme across every page and across return visits.
How many themes can I test at once?
Up to five, including your control.
Can I test my published theme against a modified copy of itself?
Yes. Use Duplicate theme in Shopify during setup, make your changes to the duplicate, and test against it.
Can I publish the variation theme while the test runs?
No. Your control must be the published theme for the experiment to work. Publishing a variation breaks it.
Can I edit a theme while the experiment is running?
Avoid it. Editing either theme mid test means you are comparing two moving targets, and the result will not describe either theme accurately.
Does Elevate publish the winning theme for me?
No. Publish it yourself in Online Store > Themes. Ending the experiment and publishing the winner are separate actions.
Will a theme test affect my SEO?
Visitors are served different themes on the same URLs, and assignment is sticky. Your URLs, page content, and site structure do not change.
Why does my theme test need longer than my other experiments?
A theme change affects every page, so its effect on any single metric is more diluted and noisier than a focused test. Expect to run for at least a full business cycle.
Something else
Contact support through the chat widget in the Elevate app or at support@elevateab.com. Include your store URL, the experiment name, both theme names, and what you expected against what happened.
Last updated