Testing versus estimation

In this section we've now seen a big part of the methodology of hypothesis testing. We will complete the picture in the next section. But before we do, we will finish off this section with a summary of the method we've seen so far. We've already compared and contrasted hypothesis testing with confidence interval estimation, but it will pay to summarize the differences between the two fields as we have seen them.

As we said earlier in this section, the most fundamental difference comes down to the fact that in a hypothesis test, we make an assumption about the population parameter we are trying to describe. In confidence interval estimation, we don't.

Conceptually a confidence interval estimate involves collecting a sample and calculating a sample statistic. We then look at this sample statistic and assert that the population parameter is 'probably not too far away from it'. This is where the interval of values comes from in the estimation.

Coin flip estimate

For example, instead of running a hypothesis test, Fred could have simply flipped the coin 400 times and recorded a sample proportion of p = 0.53 heads (for example). He could then construct an interval estimate for the population proportion, like:

0.481 ≤ π ≤ 0.579

In testing, we start out (before we even collect a sample!) by assuming the population parameter is equal to some value. Then, when we collect a sample and calculate a sample statistic, we compare the sample statistic to the proposed value for the population parameter. If they are too different, we reject the assumed value. If they are not too different, we don't reject the assumed value.

Coin flip test

As discussed earlier, a 95% hypothesis test could start by assuming the coin was fair, so π = 0.5. This sets up cut-off points of 0.451 and 0.549 for the sample proportion. That is, if Fred collects a sample and calculates a sample proportion outside this range, he will reject the assumption that the coin is fair.

So, for example, suppose he collects a sample and gets 224 heads. He then calculates a sample proportion of p = 0.56. This sample proportion is outside the range of values from 0.451 to 0.549, so he would reject the assumption that the coin is fair.

Put simply: