Road to victory

The local government had been given more than enough chances to prove itself, and the electorate was getting angry. The next election was a year away and if the government didn't do something special, it was going to lose.

'How long did you take to get to work today, Brian?' Caroline, the head of the internal policy advisory board, was onto an idea and she wanted to run it by her assistant.

'About an hour. Why?'

'Make you angry? That it takes that long?'

'Don't know. I guess.'

'What if you could get into work quicker? And home quicker at night? Would that make you happy?'

'Probably.'

After this little focus group exercise, Caroline decided that improving the city's transportation system was a splendid idea. The city has had the same system for years and it has extensive data on how long it takes a commuter to get to work under the current system. The average time was 40 minutes. Caroline wanted to propose a six month plan to overhaul the bus and train system, to bring this average down.

The proposal went through and six months later a new public transportation system was in place. (Caroline noted how this fact alone was quite remarkable in itself, given the government's history of incompetence.) But would this win the election for them? If the government could come out with an advertisement that the average commuter time had dropped below 40 minutes, this would definitely improve their chances.

Did the proposal work?

The average commuter time, μ, was 40 minutes. Until proven otherwise, this position stands. But the government would like to prove otherwise. They would like to test the claim that this average is still 40 minutes - with the hope of establishing that it is now less than 40 minutes.

Caroline is in charge of developing policy, she isn't in charge of proving that the policy works. She contacts her friend John over in the statistics department. John gets on the case and collects a random survey of 80 commuter times, and finds that the sample mean time in this sample is 36.7 minutes. John brings this new development to Caroline.

'Bring out the ad! The average time has dropped.'

'How can you be sure? What did you find?'

'I collected a sample of 80 times and got an average of only 36.7 minutes!'

'But that is just a sample. How do you know the real average has dropped?'

'Let me put it this way. If the average really was still 40 minutes, the chances of getting a sample mean this low in a sample of this size is about ... is about 0.16 percent!'

Testing a claim

Along with estimation, testing is the other major field within statistical inference. In fact, hypothesis testing is the flipside to confidence interval estimation. In estimation, we basically collect a sample and calculate a sample statistic (like a sample mean, for example). We then assert that the population parameter (like a population mean) is 'probably not too far away' from the sample statistic. The idea in testing is similar, if a little reversed.

We always start a test with some claim that has been made about a population parameter. In particular, the claim is always that the population parameter assumes some specific value. In Caroline's example above, the claim is that the population mean commuter time in the city is 40 minutes. Caroline wanted to test this claim, which is why John conducted a hypothesis test. A hypothesis test is designed to test the claim made, and it does this by collecting a sample and seeing how the sample compares to the claim.

So, in estimation, we collect a sample and then make an assertion based on what this sample 'looks like'. By comparison, in testing, we start out with an assertion and then collect a sample to test its validity based on what the sample 'looks like'.