Statistics

Every results endpoint reports the same verdict on an experiment. This page explains how to read it; the API reference covers the endpoints themselves.

Units

Rates and probabilities are percentages from 0 to 100 (3.4 means 3.4%). lift is the change against the control, also in percent. Money is in the store currency, given in currency.

Status

Elevate uses Bayesian statistics: instead of p-values, each variation gets a probability to be best on the experiment's goal. statistics.status summarizes it:

Status	Meaning
Collecting Data	Not enough visitors, conversions or days yet to judge
Trending Positive	A variation leads the control, not yet with enough certainty
Trending Negative	The control leads, not yet with enough certainty
Near Significance	The leader is close to the significance threshold
Significant	The leader has reached the significance threshold

The threshold (90% by default) and the data minimums are your store's own, from Settings → Statistical Configurations, so the status here is the status Elevate shows for the same experiment.

Winner and probabilities

/results adds the detail behind the status:

"statistics": {
  "status": "Significant",
  "winner": { "variationId": 1002, "name": "Variation 1", "probability": 99.2 },
  "probabilities": {
    "CONVERSION_RATE": [
      { "variationId": 1001, "probability": 0.8 },
      { "variationId": 1002, "probability": 99.2 }
    ]
  }
}

winner is the variation most likely to be best on the goal; it can be the control. Both fields are null while collecting data.

Goals

goal is the metric an experiment is judged on: CONVERSION_RATE, REVENUE_PER_VISITOR, AVERAGE_ORDER_VALUE, PROFIT_PER_VISITOR, ADD_TO_CART_RATE, CHECKOUT_START_RATE, CHECKOUT_COMPLETION_RATE or CUSTOM_EVENT. For CUSTOM_EVENT, the experiment's customEventTrackingId names the event.