Experiments and results

Read experiment results

Interpret descriptive totals, analysis status, variant effects, and traffic checks.

Open an experiment's Results page after mapping and syncing it with a supported testing provider. Results show the baseline, last sync time, visitor and conversion totals, overall conversion rate, Sample Ratio Mismatch, and variant effects.

A synthetic experiment report comparing a 5% baseline conversion rate with a 6% variation conversion rate, with traffic checks and variant effects.

Check readiness before the conclusion

  1. Verify the correct experiment, baseline, and primary metric.
  2. Check the last sync time and any status banner.
  3. Review visitor and conversion totals for the expected variations.
  4. Inspect Sample Ratio Mismatch before using statistical conclusions.
  5. Read the effect estimate, uncertainty, and corrected significance together.

The current statistical report uses fixed-horizon, two-sided analysis with Holm correction for the reported comparisons. It supports rate-per-user primary metrics. The planned end date must have passed and the result status must be stopped before inference is ready. Interim totals can still be descriptive.

Understand report states

Status messageWhat to do
No synced result dataMap and sync the experiment.
Result mapping is incompleteMap the primary metric and every provider variation one-to-one.
Primary metric is not supported yetCheck the metric type; fixed-horizon v1 supports rate-per-user metrics.
Fixed-horizon analysis is not readyCheck the planned end date and stopped result status.
Not enough data for a trustworthy conclusionKeep the result descriptive until the analysis checks pass.
Traffic allocation mismatch detectedInvestigate expected versus observed allocation before using significance.
Result data is staleSync again and inspect the refreshed status.
Experiment configuration cannot be analyzedCheck baseline, allocation, source, and observed counts.
Statistical analysis is temporarily unavailableKeep descriptive totals available and retry after recovery.

Interpret variant effects

A label of Significant after correction means the comparison passed the report's corrected significance check. Not significant after correction does not establish that the variations are equivalent. When a readiness or traffic check blocks inference, use the status explanation instead of reading a displayed descriptive uplift as a decision.

Record the team's decision separately on the experiment, and document the context in a learning. See result troubleshooting for a recovery checklist.

Was this article helpful?

Need more help? Copy this article's details and paste them into a request on our feedback portal. You may need to sign in.

Open feedback portal

On this page