this is a HUGE throwback here but what about jfern vs JJ debating on "the statistical significance of 1000 coinflips" or whatever
I've read through those topics and it's completely insane how long that argument went on for.
For those hopelessly confused: testing for statistical significance comes
after establishing a null hypothesis. If you null hypothesis is that the coin is fair, then a lopsided result is significant in rejecting the hypothesis. If your null hypothesis is that the coin is lopsidedly biased, a lopsided result won't significantly reject the hypothesis.
Even more loosely, if you believe a poll is biased towards Kerry, then a pro-Kerry result will not change your belief of that. This is partly why a biased sample isn't good for any statistical inference.