Statistical significance for A/B testing (using chi-square)
Npm module for calculating chi-square test that gives us p-value for statistical significance with practical use in A/B testing.
Installation
`npm install --save-dev chi-square-a-b-testing`
Usage
Let's consider we have two ad variations - each variation has been displayed 50 times. The first ad was clicked 1, whereas the second ad was clicked 5 times. Is the difference statistically significant? Let's find out using chi-square test
* * * * * * * * * * *
* SAMPLE | CLICKED *
* 50 | 1 * - ad variation 1
* 50 | 5 * - ad variation 2
* * * * * * * * * * *
var test = ; // Set up our sample valuesconst table = 50 1 50 5; // Calculate the p-valuelet pValue = ; // 0.9078770365273039
In this case the pValue = 0.9078770365273039
, which means we cannot consider the difference between those two variations statistically significant if we set our threshold to 0.5 (which is usually the case for a standard experiment).
Acknowledgement
The JavaScript implementation of chi-square test was done by http://stats.theinfo.org/ (Aaron Swartz and Ben Wikler)