The Chi-Squared Test
April 25th, 2006 by PotatoGreatly contributing to my various moments of weakness lately was the fact that I bought a box of 12 Mars bars a few weeks ago. Interestingly, they were all contest wrappers, with a 1-in-6 chance of “winning instantly” yet in 12 bars, I didn’t get a single winner. That got me to thinking that maybe they rigged the contest so that winning wrappers aren’t put into bulk packages (since the profit per bar is lower). It’s a type of manipulation that’s been done before — Tim Horton’s, for example, only has cars in it’s roll-up-the-rim contest under large or extra-large cups, and regionally skews the odds.
But rather than bandying about useless speculation and rhetoric, let’s go to the statistics! The Chi-Squared Test allows us to determine whether the observed frequency of a sample differs significantly from the expected frequency of the population. In other words, we know that in any realistic sample we’re not going to get the expected frequency of winners: for every 12 Mars bars, we won’t always see 2 winners. Sometimes there will be 1, sometimes 4, sometimes none, and if you’re really lucky, all 12 could be winners. With statistics, we can see what the odds are that our difference from the expected frequency is due to random chance alone (unfortunately, it will never tell us definitively that shenanigans are afoot — but we can take the probabilities and make up our own mind).
So, our expected frequency, E is given on the package as 1/6, or 0.1667. Our observed frequency O is 0/12, or 0. Then, we find the Chi-Squared statistic which is (O – E)2/E = 0.1667.
We do the same for the non-winning bars, O = 12/12 = 1, E = 5/6 = 0.8333. (O – E)2/E = 0.0333.
The total is 0.2000. For a single degree of freedom Chi-Squared, this is merely unfortunate, and not different enough to suspect shenanigans (a greater than 10% chance that this was due to chance alone).
But I’m keeping my eye on you, Mars-Effem, Inc.
Yes, yes I will do nearly anything to procrastinate when the mood strikes.