Experiment and A/B Test Analysis
Evaluate test results with statistical care.
The Open buttons send the prompt text as currently shown below to that site only when you click. Fill in the placeholders first.
ROLE: You are an experimentation analyst. CONTEXT: Hypothesis: [HYPOTHESIS]. Primary metric: [PRIMARY_METRIC]. Variant A: [A_VISITORS] visitors and [A_CONVERSIONS] conversions. Variant B: [B_VISITORS] visitors and [B_CONVERSIONS] conversions. Duration: [DURATION]. Secondary metrics: [SECONDARY_METRICS]. Minimum effect worth detecting: [MINIMUM_EFFECT]. STEPS: 1. Calculate conversion rates, absolute and relative lift, a confidence interval for the difference, and a p-value with the test used (show the formula). 2. Check sample ratio mismatch, novelty effects, day-of-week coverage, peeking problems, and multiple comparisons. 3. Give a decision: ship, extend, or stop, with the reasoning. 4. State the sample size that would have been needed to detect the minimum effect with 80 percent power. 5. Warn me if the data cannot support a conclusion. OUTPUT FORMAT: Calculation tables, validity checks, decision, and sample size note. QUALITY BAR: Show the arithmetic. Avoid declaring winners from underpowered tests. FOLLOW-UP: Ask me what happens to the business if we ship the wrong variant.
Fill in before use
Optional: type your details and the prompt above updates. Nothing is saved or sent.
Replace each bracketed placeholder with your own details, then paste into Any AI or another assistant.