Ship the change. Then prove it.
A/B testing and feature flags where the statistics are on screen, not behind a verdict badge. Deterministic assignment, a power calculation before launch, and a sample-ratio check on every readout — over the product analytics that define your conversions.
Revenue · Q4 to date
$1.42M
+24.6% YoYTop accounts · this quarter
Cohort retention · weekly
77%
avg W6+3.2ppAvg retention curve100% → 77%
Checkout funnel — last 30 days
+1.4pp WoWQ4 revenue · $1.31M
+22% YoYInventory aging by warehouse
Revenue
$1.31M
+18%Orders
12,408
+12%AOV
$105.6
+5%Return rate
4.1%
-0.8ppYoY growth · region × line
Q3 vs Q3How an experiment runs
From a hypothesis to a decision you can defend.
Four steps, and the arithmetic is visible at every one of them.
01 · Size it
Know how long the test needs before you start it
Give the power calculator a baseline conversion rate and the relative effect worth shipping. It returns the sample size per variant, the total, and — with your daily traffic — how many days that takes. Alpha and power are yours to set, not ours to assume.
02 · Assign
Deterministic bucketing, nothing stored
A variant is a pure function of the unit id, the experiment salt and your allocation, so a returning user lands in the same bucket without a lookup or a network round trip. Targeting decides eligibility first; experiments that share a layer stay mutually exclusive.
03 · Read
The readout shows its work
A two-proportion z-test with the p-value, z-score, the 95% confidence interval on the difference and the relative lift — all on screen, all checkable. Fixed-horizon and labelled as such, because a p-value you peek at daily is not the p-value you think it is.
04 · Trust it
A mismatch check before you believe the lift
Every readout runs a chi-square goodness-of-fit test between the exposures each variant actually received and the split you configured. Under p = 0.001, assignment or logging is broken and the result is not a result — however good the number looks.
Inputs
- Baseline rate
- 4.20%
- Relative MDE
- +5%
- Alpha
- 0.05
- Power
- 0.80
Required
139,442 per variant
≈ 18 days at 15,000 visitors/day
Relative lift
+6.81%
- Control
- 4.21% (5,872 / 139,442)
- Treatment
- 4.50% (6,272 / 139,390)
- z
- 3.7412
- p
- 0.000183
- 95% CI (diff)
- [0.0014, 0.0044]
Fixed-horizon test · read at the pre-computed N
Sample ratio OK
χ² = 0.01 · p = 0.906 · expected 50/50
Sample ratio mismatch
χ² = 214.7 · p < 0.001 · 52.1 / 47.9 observed
Readout withheld until assignment is fixed.
The check that catches the experiment that was never really running
03 · Under the hood
One platform, every data path you have.
Sources stream in, our core normalizes and stores them, and every team gets the output that fits their workflow. End-to-end encryption, per-tenant isolation, audit logs out of the box.
Warehouses
Snowflake · BigQuery · Redshift
SaaS apps
Salesforce · HubSpot · Stripe
Event streams
Kafka · Segment · SDK
Files & APIs
S3 · Webhooks · REST
InsightPro AI
Ingest · normalize · query · serve
Dashboards
Real-time, drill-down, shareable
AI analyst
Plain English Q&A, cited
Embed API
White-label for your customers
Experiments
Assignment · z-test · SRM · power
FAQ
Common questions, straight answers.
Most analytics tools report a winner and hide the arithmetic. Every readout here shows the two-proportion z-test behind it — p-value, z-score, the 95% confidence interval on the difference, and the relative lift — so a reviewer can check the call rather than take it on faith.