Statistical testing helps you know whether a difference in your data is actually meaningful — and Panoplai handles the hard part for you.
When It Appears
Stat testing runs automatically when:
- You apply Cuts to compare segments, AND
- You’re viewing results in table format
It flags significant differences using letter indicators to show which groups differ — so you can focus on insights, not calculations.
What Statistical Significance Tells You
- Whether one group’s response is truly different from another
- If that difference is statistically significant
- What the margin of error and confidence level tell you about precision
How to Interpret Results:
- Statistical Significance ≠ Importance
- A result can be statistically real — but too small to act on.
- Rule of thumb: If the difference would change your decision, it matters. If not, it’s just interesting.
- Examples:
- A 1% lift might be significant — but not meaningful
- A 10% drop in satisfaction? That’s both significant and actionable
Confidence Levels (90% vs. 95%)
Your confidence level reflects how sure you are the result falls within the margin of error:- 95% confidence = standard in most research
- 90% confidence = used for exploratory or directional analysis
The higher the confidence interval, the more certain, or confident, you can be in the statistical significance reported.
Panoplai calculates all of this automatically — so you can trust what’s real, and act on what matters.