Theme Testing Overview
What theme testing does in Elevate A/B Testing, when to use it, what you need before you start, and what to know before comparing two themes on live traffic.
Theme testing compares complete themes against each other on live traffic. One group of visitors sees your published theme, another sees an alternative, and Elevate A/B Testing reports which performed better.
Instead of publishing a new theme and hoping, you test it first and switch only when the data says to.
How it works
Your published theme is always the control. Each variation is a different theme from your Shopify theme library.
Visitors are split between them and each one browses the entire store through their assigned theme: homepage, collections, product pages, cart, and checkout. Assignment is sticky, so a visitor who returns sees the same theme as before and nobody watches your store change shape between visits.
From a visitor's point of view the store is completely normal. Cart and checkout work exactly as they always do.
You can test up to five themes at once, including your control.
When to use it
You bought or built a new theme and want proof before publishing
You want to test one page template. Use a Page Experiment.
You have done a full redesign, too large to test page by page
You want to test a headline, image, or style. Use a Content Experiment.
You are comparing design approaches, such as minimal against feature rich
You want to know which specific change caused the difference. A theme test cannot tell you.
You want to try a seasonal layout during peak season
You upgraded a theme to a new major version and want to confirm nothing regressed
A theme test answers one question: is this whole theme better than that one. If the new theme wins, you know to switch. If it loses, you will not know which of a hundred differences caused it.
Use it as a decision tool before a migration, not as an optimization tool.
What you need before you start
A finished alternative theme. Fully customized, with every section and setting configured. Half built themes lose on their own merits and teach you nothing about the design.
Apps working on that theme. Review widgets, chat, upsells, subscriptions, and anything else that injects into your storefront needs configuring on the variation theme too. Apps do not carry across automatically.
Comparable content on both. Sections, menus, and settings do not transfer between themes. If your published theme has a testimonial block and the variation does not, you are testing a testimonial block as much as a theme.
Requirements
Elevate A/B Testing installed on your store.
Your control theme published, and each variation theme present in Online Store > Themes.
Every variation theme left unpublished for the duration of the experiment.
Theme testing works on all Shopify plans.
Two things to know
The variation theme must stay unpublished while the experiment runs. The experiment depends on your control being the published theme. Publishing a variation mid test breaks it.
Some third party analytics may not track a variation theme. Tools that only fire on your published theme can under report. Elevate's own tracking covers both, but check anything you rely on separately.
What to expect
Theme tests usually need longer runs than other experiment types. They affect every page, so the effect on any single metric is smaller and noisier than a focused test.
Run for at least a full business cycle, and expect to wait rather than seeing a clear answer in the first days.
Next steps
Preview and QA a Theme Experiment, including how to preview themes without being redirected
Last updated