Troubleshooting Content Experiments and FAQs

How to read content experiment results, fixes for common editor and delivery problems in Elevate A/B Testing, and answers to frequent questions.

---

Reading the results

For what each metric means and how a winner is decided, see the reports and statistical significance documentation. Two things are specific to content experiments.

Small changes produce small effects. A restyled button will not move revenue per visitor in a week. If the change was narrow, judge it on a narrow metric and accept that the answer may be a tie.

A multi element variation is one result, not several. If you changed the headline, the button, and the section order, a win tells you the combination worked. Treat it as a decision to ship, not a lesson about headlines.

The changes do not appear on the storefront

Likely cause. The theme extension is not enabled, or the page you are viewing is not one you edited.

Fix. Confirm the extension is on, then confirm you are on the right page and in the right variation.

The change appears, then reverts

Likely cause. The theme rerenders the section after load and overwrites the change. Common on elements that update when a variant is selected.

Fix. Check the element in Preview after interacting with the page, not just on load. If the theme rebuilds that element, target a stable parent or use custom CSS instead of an element level change.

The element I selected was not the one I meant

Likely cause. Clicking usually selects the innermost element, often the text rather than its container.

Fix. Use the Layers panel to select the parent explicitly. The editor labels the selected element with its tag and class, which tells you what you have.

The layout breaks on mobile only

Likely cause. The change was made in desktop view and the mobile layout has different constraints.

Fix. Switch the editor to mobile and adjust. Check every variation at both widths before launching.

Hiding an element broke the page

Likely cause. You hid a container rather than the element inside it, so everything it held disappeared with it.

Fix. Undo, select the specific element in the Layers panel, and hide that instead.

Edited text is not saving

Likely cause. Content changes require Apply. Style changes do not.

Fix. Select Apply under the content editor after making a text change.

The variation is slower than the control

Likely cause. Custom JavaScript or CSS added to the variation, or a large added image.

Fix. Reduce what the code does on load and compress added images. A speed difference is a second change and it will affect your result.

FAQs

Do I need the theme extension? Yes. Changes are applied on the page, so the extension is required and the experiment cannot launch without it.

Do I need to duplicate anything? No. Your original page is saved automatically as the control and nothing in your theme is duplicated.

How many variations can I run? Up to five including your control.

Can one experiment cover more than one page? Yes. Use the page search in the editor to move to another page and make changes there within the same experiment.

Can I add code as well as visual changes? Yes. The editor offers Custom JavaScript and Custom CSS per variation. If the code is the point rather than a finishing touch, a custom code experiment is the better fit.

Can I add elements that are not on the page already? Yes. Text, headings, images, and containers.

Should I use content or page testing? Content edits elements on the page you already have. Page serves a different template. If you would be duplicating a template to change one element, use content.

Which Shopify plans support it? All of them.

Next steps

• Content Testing Overview

• Using the Content Editor