Preview and QA a Page Experiment
Pre-launch checklist for page experiments in Elevate A/B Testing, covering visual checks, functionality, third party apps, tracking verification, and scope.
Work through this before you launch. A broken variation does not just lose the experiment, it loses you real orders while it runs, and you learn nothing.
Every item below is pass or fail. If something fails, fix it and check again before launching.
Prerequisites: the experiment created and saved, and every variation template finished in the Shopify theme editor.
How to view a variation
Select Preview on a variation to open it as a visitor sees it. Preview renders both desktop and mobile, so you can check each variation at both widths without leaving the app.
When checking on your live storefront rather than in preview, use a fresh incognito window each time. A normal window may already hold a variation assignment from earlier testing and show you the same version every time.
Visual check
Control page
Looks exactly like your current live page, with no changes at all
Each variation
Your intended changes are present and rendered correctly
Mobile
Each variation loads correctly at phone width. Most layout breakage appears here first.
Desktop
Each variation loads correctly at full desktop width
Tablet
Checked if tablet is a meaningful share of your traffic
Side by side
The only differences between control and variation are the ones you intended. Watch for spacing, font, and missing section differences you did not mean to introduce.
Unintended differences are the most common QA failure on page experiments, because duplicating a template copies its settings at that moment and nothing after. Any theme edit you made to the original since duplicating is missing from the copy.
Functionality check
Add to cart
Adds the correct product from every variation
Navigation
Menus, breadcrumbs, and internal links work on every variation
Images
All images, thumbnails, and galleries load
Forms
Any form on the page submits correctly on every variation
Video and media
Embedded media loads and plays
Quick view and modals
Any popups or modals your theme uses open and function
Third party apps
Apps that inject into your storefront are configured per template in many themes. They do not always carry across to a duplicate.
Reviews widget
Appears on every variation
Upsell and cross sell
Any recommendation or bundle widgets appear
Subscription selector
Present and functional if the page offers subscriptions
Live chat and support widgets
Load on every variation
Trust badges and seals
Present on every variation
A missing reviews widget on one variation is not a small cosmetic difference. It is a second, unintended change, and it will move your numbers.
Tracking verification
Events are arriving
Visiting a page in the experiment produces an event
Correct variation recorded
The event records the version you actually saw
Add to cart is recorded
Adding a product produces a corresponding event
Assignment is sticky
Close the page, reopen it, and you see the same variation
Experiment data is cached and refreshes every two hours, so a test visit will not appear straight away. Select the refresh control next to Experiment Settings to pull live data.
See the reports documentation for where to find this data and what each metric means.
Template and scope check
Correct templates assigned
Each variation points at the template you intended
No 404s
Every variation loads the page, not a not found error
Theme edits saved
Any recent change in the theme editor is live on the variation
Scope is correct
If you scoped to specific products, collections, or pages, the experiment appears on those and nowhere else
Breadth is correct
If the experiment runs on all products or all collections, check several different ones, not just the first
Scope errors are quiet. An experiment running on your whole catalogue when you meant one product still produces data, and the data still looks plausible.
If your store is translated
If you use Shopify Translate and Adapt, check every variation in every language before launching. A duplicated template starts with no translations, so visitors in other languages see untranslated content until you add them.
Translated text also runs longer or shorter than the original, which can break a layout that looked correct in your default language.
The short version
If you have very little time, do this at minimum:
Open the page in a fresh incognito window.
Confirm you see one of the variations, and that it looks correct.
Add a product to cart.
Start checkout.
Confirm the events arrived against the right variation.
This covers the path that matters. If it works, the fundamentals are sound.
Next steps
Reading Results and Ending a Page Experiment
Troubleshooting Page Experiments
Last updated