Last year we reported on an article by sociologist Steve Morgan, criticizing a published paper by political scientist Diana Mutz.
A couple months later we updated with Mutz’s response to Morgan’s critique.
Finally, Morgan has published a reply to Mutz’s response to Morgan’s comments on Mutz’s paper. Here’s a passage that is of methodological interest:
I [Morgan] first offer the most important example of the repeated mistake Mutz commits, and I then explain the underlying methodology of fixed-effects models that justifies the correct interpretation. Here is the most important example of her mistake: When interpreting her models to develop a conclusion about the role of social dominance orientation (SDO) in the 2012 and 2016 elections, Mutz (2018b) states,
When a person’s desire for group dominance increased from 2012 to 2016, so did the probability of defecting to Trump. However, as shown by the insignificant interaction between SDO and wave in both analyses, there is no evidence that those high in preexisting SDO were especially likely to defect to Trump, thus countering the idea that SDO was made more salient in 2016. Instead, it is the increase in SDO, which is indicative of status threat, that corresponded to increasing positivity toward Trump. (p. 5)
Even if one accepts that Mutz’s model is perfectly specified to reveal the causal effects of interest to her, the correct interpretation of the model that she estimated would instead be this:
The causal effect of SDO on Republican thermometer advantage was 0.206 in 2012 and 0.184 in 2016. Thus, these estimates provide no evidence that SDO was more important for the 2016 election than it was for the 2012 election. In addition, the model does not reveal whether changes in individuals’ SDO levels between 2012 and 2016 caused any prospective voters to rate Trump more warmly than Romney, in comparison with Clinton and Obama, respectively.
Had Mutz recognized that such a paragraph would be the correct interpretation of her primary status-threat coefficients, she could not have written the article that she did.
Ya want the data and code? We got the data and code!
For those of you who may find the discussion of fixed-effects models useful, perhaps as an example for classroom use, here are links to some other resources:
My July 2018 Github repository, with Mutz’s code, my analysis code chunks inserted into her code, her data, and results for my original critique:
Lecture slides on fixed-effect models, using Mutz’s data that I presented at the Rostock retreat on causality:
An updated Github repository, with additional analysis code chunks, data, and results, developed for the Rostock lecture and also for my reply to Mutz’s comment: