I have been working through a draft SSRN paper on Ideological Diversity and Law School Hiring by Douglas M. Spencer and James C. Phillips of Berkeley (tip to Glenn Reynolds and Paul Caron).
The paper concludes:
Using regression analysis, we find no statistically significant relationship between
political ideology and prestige of hiring, although we identify a very large discrepancy between the proportion of new professors that can clearly be identified as liberal or conservative and those whose ideology is less clear.
There are several problems with the regression analyses in this paper:
1. I found one outright error. The indicator (dummy) variable LIBERAL is coded in this way: “Our second ideology variable is a dummy variable where 1 indicates a liberal candidate (1 to 3 on the ideology scale), and 0 indicating a conservative candidate (-3 to -1 on the ideology scale).”
That would leave those with a 0 on their ideology scale as missing data. But their regressions including this variable show a full N of 149 new professors hired. Using the statistical techniques they used, that is statistically impossible. That error needs to be corrected.
2. When coding a dummy variable, for a range of statistical reasons, the reference category should be the larger group. Thus the variable should have been coded as conservative=1, liberal or unknown=0. Using a small group as the reference group (the 0 coding) can cause havoc with the significance of that and other variables. Conservatives are not the norm; after all, only 8 professors in their sample of 149 professors were confidently identified as conservative (and about 20 more as probable conservatives). If your statistical technique treats conservatives as the norm, the regressions that result will usually lead to unusual results, unreflective of what is going on in the data.
Beyond statistical reasons for coding the dummy variable as conservatives v. others, there are the theoretical reasons as well. The usual argument is that known conservatives are discriminated against compared to more typical candidates (liberals or non-political types), not that liberals are favored over the non-political (as well as conservatives). There is a good chance that fitting the dummy variable properly will reveal a significant effect, given that the miscoded political variable’s effect is already pretty large.
3. The models have too many variables (about 20 or more variables) for the small number of subjects (149). Especially with colinearity, that can bleed off significant effects. It is unclear why particular nonlinear transformations are used. Spencer and Phillips might want to use fractional polynomials (the FRACPOLY command in STATA) to choose the best transformations for control variables (In my new article with Rafe Stolzenberg on politically motivated departure from the Supreme Court, we use fractional polynomials to fit the controls.)
4. I would try to fit the models without the race and gender variables to see whether that would make the political effect significant. That would give me some sense of how robust the conclusions are to the variables used and models fit. Then I would try interactions of race and politics to see whether the race and gender effects are really interaction effects with politics (ie, exclude FEMALE and fit CONSERV MALE, CONSERV FEMALE, and NON-CONSERV FEMALE). It wouldn’t surprise me if the female advantage is solely among non-conservative females, not spread evenly between conservative and non-conservative women. In other words, with so few cases (including only 8 clear conservatives), I would try fewer variables and different combinations to test for robustness.
UPDATE: Spencer and Phillips compute their stats using the usual assumption–that theirs is a random sample drawn from an infinite population. But in fact their sample is a systematic sample of one-third of a population of recent law teachers. If one adjusted their stats for the sampling design they actually used (which is an unusual adjustment to make), then the ideological coefficients would almost certainly already be large enough to be statistically significant. Adding a few sentences or a footnote to that effect might be a good thing.
Comments are closed.