> For the complete documentation index, see [llms.txt](https://docs.elevateab.com/elevate-helpcenter/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.elevateab.com/elevate-helpcenter/advanced-settings/statistical-configurations.md).

# Statistical Configurations

Store-wide Elevate A/B Testing settings that decide when a winner is declared, how extreme orders are capped, and how free gifts count.

***

Statistical Configurations are store-wide settings that control how Elevate A/B Testing evaluates every experiment. They decide when an experiment is allowed to declare a winner, how unusually large orders are handled, and whether free-gift items count toward revenue.

To open them, go to **Settings**, then **Statistical Configurations**.

***

### Settings at a glance

| Setting                                | Default                                              | What it controls                                                               |
| -------------------------------------- | ---------------------------------------------------- | ------------------------------------------------------------------------------ |
| **Statistical Significance threshold** | 90% (Recommended)                                    | How confident Elevate A/B Testing must be before it calls a winner             |
| **Visitors per variation**             | 500, or 2,500 for larger stores                      | Minimum visitors each variation needs                                          |
| **Conversions per variation**          | 20, or 100 for larger stores                         | Minimum conversions each variation needs                                       |
| **Minimum running days**               | 3                                                    | Minimum number of days the experiment must run                                 |
| **Exclude Test Outliers**              | On, capping orders above 3× your average order value | Caps or removes extreme order values so one large order cannot distort results |
| **Exclude Free Gifts**                 | Off                                                  | Removes $0 free-gift line items from revenue, AOV, and profit                  |

A larger store is one with more than $100,000 in order revenue over the last 30 days. New stores start with the smaller-store defaults until Elevate A/B Testing has 30 days of revenue data.

### When does an experiment declare a winner?

An experiment declares a winner only when it meets all four **Conclusive result requirements** at the same time:

1. A variation reaches the **Statistical Significance threshold**.
2. Every variation has at least the number of visitors set in **Visitors per variation**.
3. Every variation has at least the number of conversions set in **Conversions per variation**.
4. The experiment has run for at least the number of days set in **Minimum running days**.

If any one of these is not met, the experiment keeps running without a winner, even if one variation is clearly ahead. This protects you from calling a winner on early, noisy data.

Checkout experiments use lower visitor minimums, because fewer visitors reach checkout than land on the store: 40 visitors per variation, or 100 for larger stores.

### What does each conclusive result requirement do?

#### Statistical Significance threshold

The **Statistical Significance threshold** is how confident Elevate A/B Testing must be that a variation truly performs differently from the control before calling it a winner.

Options: 80%, 85%, 90%, 95%, 97%, or 99%. The recommended value is **90% (Recommended)**.

| If you              | Result                                                                                        |
| ------------------- | --------------------------------------------------------------------------------------------- |
| Raise the threshold | Fewer false winners, but experiments take longer to conclude                                  |
| Lower the threshold | Experiments conclude faster, but you are more likely to act on a result that does not hold up |

Elevate A/B Testing uses Bayesian statistics to evaluate experiments.

#### Visitors per variation

**Visitors per variation** is the minimum number of visitors each variation must receive before a winner can be called. The default is 500, or 2,500 for larger stores. It stops a winner being declared from a handful of lucky visitors.

#### Conversions per variation

**Conversions per variation** is the minimum number of conversions each variation must record before a winner can be called. The default is 20, or 100 for larger stores. A small number of conversions makes results swing easily.

#### Minimum running days

**Minimum running days** is the minimum number of days an experiment must run before a winner can be called. The default is 3. Shopper behavior changes by day of the week, so a longer minimum, such as 7 days, gives a result that covers a full weekly cycle.

#### How do I restore the defaults?

Select **Reset to Default** in the **Conclusive result requirements** panel. This restores the defaults for your store's size.

### What does Exclude Test Outliers do?

**Exclude Test Outliers** finds unusually large orders across all experiments and either caps their value or removes them, so a single large order cannot distort revenue results. It is on by default.

It has three settings:

| Setting                      | Options                                        | Default                        |
| ---------------------------- | ---------------------------------------------- | ------------------------------ |
| **Detection method**         | Multiple of AOV, Fixed order value, Percentile | Multiple of AOV (Recommended)  |
| **Cap orders above**         | Depends on the detection method, see below     | 3× your average order value    |
| **What happens to outliers** | Cap at threshold, Remove from results          | Cap at threshold (Recommended) |

#### Which detection method should I use?

**Detection method** decides how the outlier threshold is set. You enter the threshold in **Cap orders above**.

| Detection method                  | What you enter in Cap orders above                                              | An order is an outlier when                                                                               |
| --------------------------------- | ------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------- |
| **Multiple of AOV (Recommended)** | A number, such as 3. The default is 3 and the minimum is 1.                     | Its value is more than that many times your average order value. A lower number is stricter.              |
| **Fixed order value**             | An amount in your store's currency. There is no default, so you must enter one. | Its value is above that amount                                                                            |
| **Percentile**                    | 95th, 97.5th, or 99th (Recommended)                                             | Its value is above that percentile of your order values. The threshold adapts as your order data changes. |

**Multiple of AOV** suits most stores, because the threshold moves with your average order value. Use **Fixed order value** if you know exactly which order size is unusual for your store.

#### What happens to an outlier order?

| Option                             | Order value              | Counts as a conversion                                             |
| ---------------------------------- | ------------------------ | ------------------------------------------------------------------ |
| **Cap at threshold (Recommended)** | Reduced to the threshold | Yes                                                                |
| **Remove from results**            | Excluded                 | No. The order is excluded from all metrics, including conversions. |

**Example:** your average order value is $100 and **Cap orders above** is 3, so the threshold is $300. A visitor places a $2,000 order. With **Cap at threshold**, it counts as a $300 order and one conversion. With **Remove from results**, it does not count at all.

**Cap at threshold** is the safer choice for most stores, because the visitor still converted and that conversion is real. Keep **Exclude Test Outliers** on unless very large orders are a normal part of your business, such as wholesale orders.

### What does Exclude Free Gifts do?

**Exclude Free Gifts** removes free-gift line items added by bundle and gift-with-purchase apps from experiment calculations. These $0 items inflate units sold and skew results. It is off by default. Turn it on if you use a bundle or gift-with-purchase app.

When it is on:

* Free-gift line items are removed from revenue, average order value (AOV), and profit calculations.
* Orders that contain free gifts still count as conversions.

### FAQs

**Why has my experiment not declared a winner even though one variation is ahead?** At least one of the four **Conclusive result requirements** is not met yet. Check visitors and conversions per variation, and how many days the experiment has run.

**Do these settings apply to every experiment?** Yes. Statistical Configurations are store-wide.

**Does Exclude Test Outliers remove large orders from my results?** It depends on **What happens to outliers**. With **Cap at threshold (Recommended)**, a large order still counts as a conversion and only its value is reduced. With **Remove from results**, the order is excluded from all metrics, including conversions.

**Does Exclude Free Gifts remove orders that include a free gift?** No. Only the $0 free-gift line items are removed from revenue, AOV, and profit. The order still counts as a conversion.

### Related

* [Experiment and Store Settings](/elevate-helpcenter/advanced-settings/experiment-and-store-settings.md) for experiment, tracking, and store behavior settings