Here’s question 8 of our exam:
8. Out of a random sample of 50 Americans, zero report having ever held political office. From this information, give a 95% confidence interval for the proportion of Americans who have ever held political office.
And the solution to question 7:
7. You conduct an experiment in which some people get a special get-out-the-vote message and others do not. Then you follow up with a sample, after the election, to see if they voted. If you follow up with 500 people, how large an effect would you be able to detect so that, if the result had the expected outcome, the observed difference would be statistically significant?
Assume 250 got the treatment and 250 got the control. Then the standard error of the estimated treatment effect is sqrt(0.5^2/250 + 0.5^2/250) = 0.045. An estimate is statistically significant if it is at least 2 standard errors from 0, so the answer to the question is 0.09, an effect of 9 percentage points.
Most of the students couldn’t handle this one. One problem was forgivable: I didn’t actually say that half the people got the treatment and half got the control. I guess I should’ve made that clear in the statement of the problem.
But that wasn’t the only issue. Many of the students weren’t clear on how to get started on this one. One key point is that you can plug p=0.5 into the sqrt(p*(1-p)/n) formula.