Create a Checkout Experiment
Step by step setup for a Checkout Experiment in Elevate A/B Testing, covering all seven creation steps, enabling the checkout extension in Shopify, and verifying the experiment after launch.
This page walks through the seven steps of the New Checkout Experiment flow, the checkout extension you enable afterwards, and how to verify the experiment once it is live.
Before you start
Shopify Plus
Checkout Experiments run on Shopify Plus only. This is a Shopify platform restriction.
Checkout Editor access
You need permission to edit the checkout in Shopify, because each component's extension is enabled there.
Variations
Up to five, including the control.
A hypothesis
Know which component you are adding or changing and what you expect it to do. Checkout traffic is small, so a vague test rarely produces a readable result.
There is no live preview before launch. The preview panel in the app renders the component as it will appear, but the only way to see the experiment on your real checkout is to launch it and start a checkout yourself. Plan for that before you launch: see Preview and QA Checkout Experiments.
Step 1: General Info
Go to Experiments and select New Experiment, then choose the Checkout Experiment type.
Enter an Experiment name. This is the only required field. Name it for what it tests, such as "Checkout trust badges vs none," so it is readable in your results list months later.
Add a Description and a Hypothesis if you want them. Both are optional. The hypothesis is the field you will thank yourself for when you read the results.
Add Assets if useful. You can drop in images and documents up to 10 MB and videos up to 100 MB, for reference material such as mockups.
Under What do you want to measure?, select your experiment goal:
Revenue per visitor
The default recommendation. It captures both conversion and order value in one number.
Conversion rate
You care only about completing more checkouts, not about order size.
Average order value
You are testing an upsell or anything meant to increase basket size.
Profit per visitor
You want margin rather than revenue. It is always selectable, but only choose it if you have added GOGs. Without cost data the profit figure will not reflect your real margin.
Select Browse all metrics for the full list. The goal decides how the winner is chosen. Every other metric is still calculated and shown in your results.
Select Continue.
Step 2: What component do you want to test?
Select one or more components. Each card shows a rendering of what that component looks like in checkout.
Checkout Banner
The main banner at the top of checkout, with a heading and description
Trust & Security Badges
Trust signals and security certifications
Accepted Payment Methods
A titled row or column of payment icons
Checkout Upsell
Product offers shown during checkout
Customer Testimonials
Customer reviews with star ratings
This selection is not binding. Every component appears again in Step 3, and you can toggle one on there without coming back.
If what you want to test is not listed, use Don't see the component you need? to send it to the team. Requests feed the development priority list, and several components are already in progress. See Checkout Testing Overview for what is coming.
Select Continue.
Step 3: Configure checkout
This is the main setup step. The tabs at the top are your variations: Control is A, Variation 1 is B, and the plus button adds another, up to five including the control.
Each component has a toggle and an expandable panel. Turn on the components you want in that variation and configure them. The Preview panel on the right shows the component you are editing, and reads "Select a component to preview" until you open one.
Two things to understand before configuring:
The control is configurable. Leave it with everything off to test a component against a checkout with nothing added, or configure it to test one version of a component against another.
The preview is a rendering, not your checkout. It shows the component's own appearance. It does not show the component in place on your real checkout page.
Checkout Banner
Banner Type
The banner's colour and icon treatment: Auto, Info, Success, Critical, or Warning
Spacing
How much space the banner takes above and below
Heading Text
The bold first line, with its own alignment control
Description Text
The supporting line beneath it, with its own alignment control
Match the banner type to the message. Critical and Warning carry alarm colouring, which works for genuine scarcity and works against you for a friendly reassurance message.
Trust & Security Badges
Display Settings
Vertical stacks the badges, Horizontal places them in a row
Alignment
How the group sits within its space
Padding
Space around the group
Badges
Up to three badges, each with an icon, a title, and a description
Use the pencil to edit a badge, the bin to remove one, the drag handle to reorder, and Add a new badge to add one. Order matters: the first badge takes the strongest position in both layouts.
Accepted Payment Methods
Section Title
The line above the icons, with alignment, size, and colour controls
Icons Alignment
How the icon row sits within its space
Payment Icons
The methods shown, in order
Reorder icons with the drag handle and add more with Add More Payment methods. There is no three item cap here. List the methods your customers actually use, and put the ones that reassure them first.
Checkout Upsell
Offered Products
Auto Recommended Products uses Shopify's recommendation system. Offer Specific Products lets you choose them yourself.
Layout
Vertical or Horizontal
Number of products
Up to three offers
Title and Subtitle
The heading and supporting line, each with alignment, size, and colour controls
Button text
The label on the add button
Display button with outline style
Switches the button from solid to outlined
Show compare at price
Shows the crossed out original price where one is set
Load new offers after an offer is accepted
Replaces an accepted offer with another product
Keep product visible after adding to cart
Leaves an accepted product in the list rather than removing it
Confirmation message
Shown once every offered product has been added to the cart
If you are testing an upsell, set your goal to Average order value or Revenue per visitor in Step 1. Measuring an upsell by conversion rate alone will miss the effect it was built to produce.
Customer Testimonials
Layout
Stacked shows reviews one under another, Carousel rotates them
Star Icon Size
The size of the rating stars
Star Icon Tone
The colour treatment of the stars
Number of visible reviews
How many show at once
Testimonials
Up to three, each with a Review Title, Reviewer Name, Rating, and Review Text
Use real reviews. A testimonial that reads as invented undercuts the trust the component exists to build.
Select Continue when every variation is configured.
Step 4: Traffic Allocation
Use the slider to set how much traffic each variation receives. The default is an even split, and the percentages must total 100.
Isolate traffic for this experiment is optional and off by default. Turning it on dedicates a percentage of your traffic exclusively to this experiment, so those visitors join no other experiment. See Traffic Isolation for when that is worth the traffic it costs.
Select Continue.
Step 5: Audiences
Narrow who is eligible, or leave everything at its default to include all visitors.
Device Type
All Devices, Desktop, Tablet, Mobile
Visitor Type
All Visitors, New Visitors Only, Returning Visitors Only
UTM Parameters
utm_source, utm_medium, utm_campaign, utm_content
Traffic
Source and Country
Create Custom Audience builds a reusable segment. See Audience Targeting for the full set of options.
Be conservative here. A Checkout Experiment already sees only the visitors who reached checkout, and every filter you add cuts that further. Narrow the audience only when the hypothesis genuinely depends on it, such as a mobile only trust message.
Select Continue.
Step 6: Enhance Your Variations (Optional)
Your variations are already set at this point. These three toggles layer additional changes on top, and most checkout experiments need none of them.
Content Editing Changes
You also want to change text or images elsewhere per variation
Custom Code
You need JavaScript or CSS changes per variation
Click-Through Rate Tracking
You want to track clicks on specific buttons or links
Skip anything you do not need and select Continue.
Step 7: Review Experiment
Check the experiment name, goal, and every variation's configuration. Then select Create Preview Experiment.
The experiment is created but not live. It moves to the Experiment Review screen, where you enable the checkout extension.
Enable the checkout extension
Shopify requires each component's extension to be switched on in the Checkout Editor before it can render. This is a per component, per store step: once you have enabled the Checkout Banner extension on your store, later experiments using the Checkout Banner skip it. A component you have never used before needs enabling the first time.
On the Enable Checkout Extension screen, select Enable Extension in Shopify. This opens the Shopify Checkout Editor.
Find the Elevate A/B Testing extensions for this experiment in the extensions list.
Enable the extension and save.
Return to Elevate A/B Testing and select I Enabled the Checkout extension.
Select Launch Experiment.
A walkthrough video sits beside these steps on the same screen if you would rather watch it.
If you skip this, the experiment can be created but the components will not appear at checkout for anyone.
Related
Checkout Testing Overview: what this type can and cannot test, and how checkout metrics are counted.
Preview and QA Checkout Experiments: how to verify each variation renders correctly.
Ending an Experiment: how to end a Checkout Experiment and what happens afterwards.
Last updated