Troubleshooting Split URL Experiments and FAQs
How to read split URL experiment results, fixes for common redirect and URL matching problems in Elevate A/B Testing, and answers to frequent questions.
Reading the results
For what each metric means and how a winner is decided, see the reports and statistical significance documentation. Three things are specific to split URL.
A win is a page, not a reason. Two entirely different pages produce one answer: which one sold better. If you want to know why, that is a follow up content or page experiment on the winner.
Check the pages were comparable. A variation that loads more slowly, or is missing an app the control has, is being judged on that as much as on its design.
Watch traffic sources. If the control URL receives paid traffic and the variation does not normally, or the two pages are indexed differently in search, the visitors reaching each page may not be alike.
No redirect happens
Likely cause. The incoming URL does not match the control, usually because of a trailing slash, a query parameter, or a strict Matches exactly condition.
Fix. Loosen the URL match condition to Starts with or Contains, or remove parameters from the control URL. Confirm the experiment is running rather than in Draft or Preview.
Visitors are redirected from pages that are not in the experiment
Likely cause. A Contains, Starts with, or Regex condition is broader than intended.
Fix. Tighten the condition. Test pages that should be excluded before relaunching.
Visitors are redirected every time and it is disruptive
Likely cause. All Experiment URLs is selected, which redirects on every landing.
Fix. Switch to Redirect once or Redirect once control URL only if repeated redirects are the problem, for example when visitors bookmark a variation URL.
The variation URL returns a 404
Likely cause. The page was unpublished, renamed, or deleted after the experiment was set up.
Fix. Restore the page or edit the experiment to point at a URL that exists. Check this before launching, not after.
Results are lopsided in a way traffic allocation does not explain
Likely cause. One URL is receiving direct traffic that never passes through the redirect, from ads, email, search, or bookmarks.
Fix. Check where traffic to each URL is coming from. Consider All Experiment URLs so visitors landing directly on a variation are still assigned properly.
An external page will not work
Likely cause. Only URLs on your Shopify store are supported.
Fix. Move the page onto your Shopify storefront, or use a different approach.
The experiment runs but conversions are not attributed
Likely cause. The purchase happened on a page or flow outside the experiment's URLs.
Fix. Confirm the full path from every experiment URL through to checkout produces events, and that all relevant pages are included in the experiment.
FAQs
Does split URL testing need the theme extension? No. The visitor is sent to a different URL rather than having the page changed, so the extension is not part of the delivery path.
Can I test a page hosted outside Shopify? No. All URLs must be on your Shopify store, because pages outside it cannot be tracked.
Can the control have more than one URL? Yes. Select Add another URL on the control card. Each variation takes a single URL.
Will visitors see the URL change? Yes. A redirect is a navigation, so the variation URL is what appears in the address bar and what a visitor bookmarks or shares.
What is the difference from page testing? A page experiment serves a different template at the same URL. A split URL experiment sends the visitor to a different URL. If the alternative page already exists at its own address, use split URL.
Can I add content or code changes on top? Yes. Step 5, Enhance Your Variations, offers content editing, custom code, and click-through rate tracking. Adding them makes the experiment multivariate, so the result tells you the combination won rather than which part did.
Which Shopify plans support it? All of them.
Next steps
Last updated