Troubleshooting Product Image Experiments and FAQs

How to read product image experiment results, fixes for common image swap problems in Elevate A/B Testing, and answers to frequent questions.

---

Reading the results

For what each metric means and how a winner is decided, see the reports and statistical significance documentation. Three things are specific to image experiments.

Watch returns, not just conversions. An image that flatters the product will lift add to cart and can raise returns with it. A win on conversion rate that costs you on returns is not a win, and the experiment will not tell you that on its own.

Expect the effect to be larger on mobile. The gallery dominates a phone screen. If your results are flat overall, look at the split by device before concluding images do not matter for your store.

A tie is a real answer. If two image sets perform the same, you have learned that the more expensive shoot is not worth it. That is worth knowing.

The images did not change

Likely cause. The theme extension is not enabled, or the product is not in the experiment.

Fix. Confirm the extension is on, then check the product is in the selected set.

Images changed on some products but not others

Likely cause. Products with different image counts or variant linked galleries can behave differently on some themes.

Fix. Check the affected products individually in Preview. Themes that tie images to variants are the usual cause.

The variation image looks stretched or badly cropped

Likely cause. The replacement image has different dimensions or an aspect ratio your theme does not handle the same way.

Fix. Match the dimensions and aspect ratio of your control images unless the format itself is the thing you are testing.

The wrong image is showing first

Likely cause. The image order in the variation is not what you intended, or the intended image is not set as Default Image.

Fix. Reopen the experiment and drag the images into the correct order.

The image changed on the product page but not on collection pages

Likely cause. Some themes and apps read the product's featured image directly rather than through the storefront, so they show your original.

Fix. Nothing to fix, but account for it when reading results. Your experiment is testing the product page image only in that case.

Variant selection shows the original image

Likely cause. Your theme links images to product variants and reloads the gallery on selection.

Fix. Check this in Preview before launching. If the theme overrides the swap on variant change, the experiment will not measure what you intended for those products.

FAQs

Does this change my product images in Shopify? No. Images are swapped on the storefront only. Your Shopify product data is untouched, and your original images return when the experiment ends.

Do I need to duplicate products? No.

How many products can one experiment include? Up to 500.

Does it need the theme extension? Yes. The swap happens on the page, so the extension is required and the experiment cannot launch without it.

Can I use AI to create the test images? Yes. Replace Image includes a Generate with AI tab that generates from the existing product image and a description. Review the result carefully before using it, since a misleading image causes returns.

Can I test image order without replacing anything? Yes. Drag images into a different order in the variation and leave the images themselves alone.

Which Shopify plans support it? All of them.

Next steps

• Product Image Testing Overview

• Create a Product Image Experiment