Paul W. answered • 10d
PhD in Statistics with 20+ Years of Math / Stat Education Experience
Very interesting question! I'll see if I can give you a good answer.
In polling voter opinions about a school bond issue, two groups randomly sampled the population of river city. both polls have defined their population parameter to be the proportion, null hypothesis, of voters who plan to vote "yes" , and both groups have chosen to conduct an hypothesis test of the null hypothesis: p=.94 against a two-sided alternative. However, the research department rejected the null hypothesis, while the voter's group failed to reject the null. Which of the reasons below might explain this discrepancy?
A)The voters' group used a higher alpha value.
B) The voters' group is reporting with a smaller sample size.
C) the voters' group has a larger estimated standard deviation for the sampling distribution of p.
D) The voters' group has a smaple proportion, p, that is farther from .84 than that for the research group.
What we know is that both groups assumed that the true proportion of "yes" voters is 94%. The problem suggests that there was a test comparing the proportion of "yes" voters to the null value of 0.94. While we don't know what was observed in the two different samples, we do know that one sample (the voters' group) produced an estimate that was statistically different from 94% while the other (the research group) was not.
There is enough missing information in the problem to make any of the reasons for the difference seem credible. Let's go through them one at a time and see if they make sense.
A)The voters' group used a higher alpha value.
A larger alpha would make it easier to reject the null hypothesis. Let's assume both groups got the same proportion estimate from their sample and they had the same number of people in each sample. The critical value for the Z-test comparing the sample proportion to the null is based on alpha only so we can look at those values directly:
Alpha C (two-sided)
0.01 2.576
0.05 1.96
0.10 0.84
Let's say that the research group used an alpha of 1% but the voters' group used an alpha of 5%. Even though both groups would have the same test statistic based on their sample, if the statistic fell between 2 and 2.5 then we would get different conclusions from the two different groups. This means that (A) could be a good reason for the discrepancy.
B) The voters' group is reporting with a smaller sample size.
If we assume that the two groups got the same proportion but had samples of different size then it is not possible for the significant result to go with the smaller sample size. The test statistic is:
Z = (p - p0)/sqrt( p0 * (1-p0) / n)
Which may be restated as Z = sqrt(n) * f(p,p0)
Since we are assuming p and p0 are the same between the studies f(.) is a constant and the only difference between the two studies would be sqrt(n). Since rejecting happens when Z is larger than the established critical value for a fixed alpha, anything that makes Z bigger is more likely to produce a significant finding. This means that if there's a difference in conclusions we're more likely to see significance from the larger sample. So, (B) cannot be the correct answer because the voters' group got a significant result if everything else about the data were the same and only the sample size was different.
C) the voters' group has a larger estimated standard deviation for the sampling distribution of p.
This is a similar kind of answer to what made (B) invalid as a reason. The sampling SD is the denominator of the test statistic. If the denominator gets larger than the Z gets smaller and we probably won't reject the null. The only way for the sampling SD estimate to be smaller is if the n is smaller since p and p0 are assumed to be the same between the studies.
D) The voters' group has a smaple proportion, p, that is farther from .84 than that for the research group.
It's not clear why the problem changed to 0.84 when the null was set at 0.94. If this is a typo and the OP meant to use 0.94 here then this is definitely a good option. The test statistic talks about how far away the sample proportion is from the null value in standard units. The standardizing value is sqrt(0.94 * 0.06/n) where n is the size of the sample. If n is the same for both studies then a sample estimator that is further away from the null is going to have a larger standardized difference and a better chance of producing a significant result. This means that (D) is also a good reason for why the two groups got different results.
So, my final answer is that (A) different alphas and (D) different sample estimates are both good reasons for why the groups got different results.
