Assuming the null hypothesis is true
Why is this so important? Let's look at the example of Fred's coin that may or may not be weighted.
Fair coin?
Recall Fred had a coin that he suspects of being weighted. He wants to flip the coin 400 times and run a hypothesis test so that he can test the claim that the coin is fair. That is, he wants to test the null hypothesis:
H0: π = 0.5
His alternative hypothesis is that the coin is not fair:
HA: π ≠ 0.5
So under the methodology of hypothesis testing, he would start by assuming that the coin is fair. That is, the proportion of times that a head shows up is 50%. In other words, for the purposes of running the test he would assume that π = 0.5.
So where does an assumption like this get us?
Put simply, it tells us what to expect out of any sample we might collect. And if the sample we do collect goes too far against this expectation, we will reject the assumption - and reject the null hypothesis.
But we can be more precise than this by looking at sampling distributions. Assuming that the population proportion π is 0.5 can actually tell us quite a bit about what to 'expect'. Think about the sampling distribution of the proportion, P.
Recall that P approximately follows the normal distribution with mean π and standard deviation (π(1 - π)/n)1/2. But under the assumption that the null hypothesis is correct, we know π is 0.5. And if Fred flips the coin 400 times, we know n = 400. So actually, we can say that P approximately follows the normal distribution with mean 0.5 and standard deviation 0.025.
We've gotten quite far with that assumption!
And what do we know about the normal distribution? We know things like:
95% of all values fall within 1.96 standard deviations of the mean
Relating this to the sampling distribution, we can say that:
For 95% of all samples (of size 400), the sample proportion will be within 1.96 × 0.025 = 0.049 of the population proportion 0.5.
In other words, making the assumption that the null hypothesis is correct, we can say that 95% of all samples will give a sample proportion somewhere between 0.451 and 0.549. (Relating this to the physical number of heads and tails, this means that 95% of the time that a sample of 400 coin flips is observed, the number of heads will be between 400 × 0.451 = 180.4 and 400 × 0.549 = 219.6.)
Again, all of this comes from the simple assumption that the coin is fair. So you might say that, since we are assuming that the null hypothesis is true, we 'expect' to collect a sample with a sample proportion between 0.451 and 0.549.
