pageok
pageok
pageok
Miles and Sunstein's Response to Their Critics:

Thomas Miles and Cass Sunstein have now responded to critics of their study of judicial "activism" and ideological bias in Supreme Court review of administrative agency decisions. To briefly recap the Miles-Sunstein methodology, they code as "activist" justices' votes to overrule agency decisions instead of defer to them, and they characterize the justices as ideologically "partisan" if they are more likely to vote to strike down "conservative" agency decisions than "liberal" ones or vice versa.

In an earlier post, which Miles and Sunstein kindly cite in their reply, I criticized their approach on three grounds: that it fails to control for the underlying distribution of agency decisions (ignoring the possibility that an agency may make more flawed "conservative" decisions than liberal ones, or vice versa); that it ignored the fact that some justices may have centrist ideologies that skew their decisions just as much liberalism or conservative ones; and that their praise of Justice Breyer as the most "restrained" and "humble" justice may be misplaced in so far as he may actually have ideological reasons for maximizing the autonomy of agency bureaucrats.

Miles and Sunstein respond to the first of these points as follows:

If the distribution of agency decisions were skewed in a liberal direction, as some critics allege, we should have observed few or even no challenges from public interest groups. Instead, we observed a fair number of such challenges. Moreover, our study period included many decisions from both the Clinton and the Bush administration, and it would be a big surprise if decisions by the latter were mostly "liberal."

I don't think that if agency decisions were skewed in a liberal direction, it necessarily follows that there would be "few or no challenges from [liberal] public interest groups." The agency could skew in a liberal direction, but the public interest groups might want it to be skewed that way even more. Moreover, an agency skewed in a liberal direction may be more likely to be influenced by challenges from liberal groups to change its policies than one that is conservative. After all, a liberal agency is more likely to sympathize with the agenda of outside liberal critics. Thus, far from preventing challenges to its actions by liberal groups, a liberal-biased agency might actually attract them.

Moreover, the issue in question is not so much whether an agency makes "mostly liberal" decisions as whether it is more likely to err in its interpretation of the law in a liberal direction than a conservative one. If it does, than justices who are more likely to overrule liberal decisions than conservative ones may not be acting in a "partisan" manner, as Miles and Sunstein contend. For example, it's theoretically possible that an agency makes 80% "conservative" decisions, but that half of the 20% that are "liberal" are misinterpretations of the law, while none of the conservative ones are. In such a scenario, the agency makes many more conservative than liberal decisions, but the direction of its errors would still be skewed in a liberal direction. Contrariwise, we might get the opposite result if an agency is more likely to make conservative errors tahn liberal ones.

I am also not convinced that including the Bush administration necessarily protects against the possibility that erroneous agency decisions will skew in a liberal direction. The Bush Administration is a "big government conservative" group that has not paid much attention to regulatory reform, and that has strongly supported massive increases in domestic spending. It would not be surprising if in many agencies, a lot of decisions were left to the permanent bureaucracy (which is often liberal) and if many of those decisions would tend to skew liberal. Such a skew might even be acceptable to those Bush political appointees who approve of the massive growth of government that the Administration has presided over and (for the most part) supported. Probably, any such skew would be smaller than in a Democratic administration, but it may not be nonexistent.

I should emphasize that there are also theories that suggest that agency decisions are systematically skewed in a conservative direction rather than a liberal one. The Miles-Sunstein framework fails to address those theories either.

Lastly, I should note another problem with the Miles-Sunstein categorization scheme. As they note in their response to critics, "[a]gency decisions challenged by industry were deemed liberal, and those challenged by public interest groups were coded conservative." Since Adam Smith, free market advocates have recognized that business interests often favor government regulation when it suits their self-interested purposes. For example, businesses often favor regulations that hobble their competitors. Thus, the fact that an agency decision was challenged by "industry" doesn't necessarily mean that it was "liberal" (at least in the sense of being anti-free market). Similarly, a decision challenged by public interest groups isn't necessarily "conservative." There are now many libertarian and conservative public interest groups such as the Washington Legal Foundation that often challenge agency actions. As Miles and Sunstein correctly note, their coding methodology here is fairly standard in the field. But that doesn't mean that it is unproblematic.

UPDATE: Ed Whelan of the Ethics and Public Policy Center has also written a rejoinder to Miles and Sunstein's reply to his original critique of their study. See here.

frankcross (mail):
This is certainly a possibility. Or it might be opposite. But the cases do not arise randomly. They are self-selected on cert. So this criticism relies not on a claim that agencies generally decide too liberally but on a claim that the Court systematically takes certiorari on cases where the agency is disproportionately too liberal. Which is possible but would benefit from a theory.

On the second point, my convention is not to code business challenges as conservative if there are other businesses on the other side of the case, to account for rent-seeking litigation. I don't know if Sunstein and Miles did this. I would hope the coded challenges by conservative public interest groups differently from those by liberal groups.
11.1.2007 6:37pm
mcallen (mail):
How did Miles-Sunstein categorize FDA v. Brown and Williamson? I think that most people would think that the FDA's action was "liberal" and that Bryer was being "liberal" in his (very well written) dissent defending the FDA's decision to regulate cigarettes. But only someone with little experience in administrative law would consider Bryer's opinion to be "restrained" or "humble". Clearly, it represented support for a hype-activist role for both the agency and the courts as it would have allowed the agency to take a position that Congress had rejected.

-mark
11.1.2007 7:12pm
Ilya Somin:
This is certainly a possibility. Or it might be opposite. But the cases do not arise randomly. They are self-selected on cert. So this criticism relies not on a claim that agencies generally decide too liberally but on a claim that the Court systematically takes certiorari on cases where the agency is disproportionately too liberal. Which is possible but would benefit from a theory.

That's a good point. But the Court could well end up doing this if there is a conservative majority on the Court (as there has been for most of the period studied by Miles and Sunstein). Also, the agency's own biases could affect the pool of cases available to take cert on.
11.1.2007 7:25pm
eric (mail):
I think the best thing that could happen is that they release their characterization of each case in the study and explain if they use a per se rule on the public interest group - industry distinction.

That would be useful information and would reveal if this study tell us something or nothing.
11.1.2007 7:52pm
Adam J:
I agree that the coding methodology is flawed, but do you have a better method? Wouldn't it be worse for the researchers have a more nuanced coding method would likely give them alot of discretion? I think you need some kind of standard that gives the researcher too much discretion in choosing if a case is liberal or conservative. e.g. (gee this case is challenged by an industry group, but I think that they're just protecting their market so we won't code this as a liberal agency decision)
11.1.2007 8:04pm
eric (mail):
Adam J,

The lack of discretion is good, but it does not mean anything unless the "objective criteria" is objectively correct. I could decide if the case was liberal or conservative based on whether the last letter in the opinion came before m or not. 100% objective, but not accurate at all.
11.1.2007 8:08pm
eric (mail):
To clarify my last post. I think it is only fair that they should disclose the underlying data so it can be criticized. It will only make their argument stronger if they are right.
11.1.2007 8:18pm
anym_avey (mail):
I agree that the coding methodology is flawed, but do you have a better method?

A critic need not have an alternate methodology available in order to observe the problems of the existing one. Obviously it is helpful if they do, but not required. More to the point, what is the purpose of having a methodology in the first place? It should produce an outcome that at least narrows the formulative question.

If the methodology is executed and criticism can reduce the outcome to "Someone is interested in this topic" and "Sometimes the topic produces results this way, and sometimes it produces results the other way", then it is not so much a methodology as it is a circular way of returning to the formulative questions. It suggests the hypothesis is too general and should have been narrowed further prior to testing.
11.1.2007 8:34pm
frankcross (mail):
Ilya, I've been trying to figure out the selection effect here. On your first hypothesis, which I've considered, you couldn't tell as much from the votes, but the justices would show functionally the same ideological bias, just at the certiorari level. The agency biases would affect the pool mainly if the bad decisions were upheld at the circuit court level. And really bad ones probably won't be so upheld.

But my belief is that justices take cases on cert because (a) they are very close questions on the law and/or (b) they are of great practical importance. If this is true, (a) shouldn't show any biasing effect but (b) might have some such effect. A bad liberal agency decision might have a bigger effect than a bad conservative one. However, even then, the Court would only need take it if (usually) the DC circuit upheld the bad liberal decision, and that circuit was pretty conservative during this time period. There's surely some selection effect bias here, but intuitively it seems like it would be very small.

If you believe that the Court generally takes cases that are very close on the law, the ideological findings aren't terribly exceptional, showing just an effect in fairly marginal cases.
11.1.2007 8:41pm
Adam J:
anym_avey- Sure, you don't need an alternate methodology, but if you don't suggest an alternate methodology you haven't necessarily accomplished much- frequently the best tactics (think capitalism, democracy, etc.) still have significant flaws &drawbacks. Somin implies that they used a poor coding method because it's flawed, but maybe it's the best coding method that could have been used, flaws and all.

Also, I'm not sure I understand your second paragraph... but Somin has only shown that the results are flawed in specific situations, but he has hardly shown they are useless
11.1.2007 8:53pm
eric (mail):
Here is an interesting example of the problem:

National Cable &Telecommunications Ass'n v. Brand X Internet, 545 U.S. 967 (2005). The majority per Thomas rejected an industry challenge to an FCC ruling that cable company provided broadband Internet service is not a "telecommunications service" and therefore not subject to certain regulations.

Scalia, Souter and Ginsburg dissented.

So Scalia, Souter and Ginsburg were being conservative activists? Thomas et al were being restrained liberals.

Of course, industry was on both sides of this case. It is hard to characterize as conservative or liberal. I do not know if FCC cases were included in their analysis. Either way, this case seems illustrative of the problem.
11.1.2007 9:00pm
Ilya Somin:
Somin has only shown that the results are flawed in specific situations, but he has hardly shown they are useless

I don't claim that they are completely useless, merely that they don't prove nearly as much as the authors contend. I'm more than willing to admit that the results have SOME utility.
11.1.2007 10:03pm
Larry Fafarman (mail) (www):
The term "activist judge" is one of the most misused terms in jurisprudence. It has been used as a code term that means "liberal judge" and as a code term that means "conservative judge." Originalism, which is supposed to be extremely conservative, has been called a form of judicial activism. In his Kitzmiller v. Dover opinion, Judge John E. Jones III said,

Those who disagree with our holding will likely mark it as the product of an activist judge. If so, they will have erred as this is manifestly not an activist Court.

However, later, in an interview, he said, "People term 'activist judges' judges they don't agree with." So what he meant in his above statement in the Kitzmiller opinion was, "I'm a judge that people don't disagree with."

IMO we should stop using the terms "activist judge" and "judicial activism" because there is no consensus on what these terms mean.
11.1.2007 10:13pm
Adam J:
"I'm more than willing to admit that the results have SOME utility." -spoken like a lawyer, willing to concede as much as possible so long as your case is still made. However, flaws and all, I think it's significant evidence that liberal judges are not the only one deciding cases based on ideology.
11.1.2007 10:34pm
Eli Rabett (www):
It's not the coding, it's the concept. Activist is used as a code word by the right for judges whose decisions they disagreed with. Since things were just fine in 1850, anything which changed the status quo antebellum was activism.
11.1.2007 11:20pm
Christopher Cooke (mail):
Ilya's criticisms strike me as entirely theoretical, but not likely to be correct in the vast majority of instances. For example, his comment that measuring an agency's decision as liberal or conservative based on whether it is challenged by a public interest group is not reliable because liberal groups might challenge decisions anyway, is certainly a theoretical possibility but how likely is it? Can he cite examples of such cases? If not, we are left to ask, who is likely to be right in most instances--Sunstein and Miles or Somin. I go with the Chicago profs, unless someone can cite (1) decisions coded "conservative" by Sunstein/Miles by this criteria which are objectively "liberal."
11.1.2007 11:48pm
SIG357:
If not, we are left to ask, who is likely to be right in most instances--Sunstein and Miles or Somin. I go with the Chicago profs, unless someone can cite (1) decisions coded "conservative" by Sunstein/Miles by this criteria which are objectively "liberal."




Since the Chicago profs have not yet revealed the cases they used, it's impossible for anyone to respond to your challenge.

In any case their methodology looks suspect. An activist judge is not one who overturns decisions made by a legislature or the people via a ballot inititive, but one who restricts the authority of federal bureaucrats? Really? Come on now!

It will be interesting to see if Massachusetts v. EPA was one of the rulings Sunstein used.

Bear in mind that in that case, the liberal justices insisted that the EPA had authority which the EPA denied having. How would Sunstein score that one?
11.2.2007 8:16pm
Ralph Phelan (mail):
"(link)Adam J:
I agree that the coding methodology is flawed, but do you have a better method?"

Given that the current method produces meaningless results, I'd say its incumbent on the researchers to either find a better method, or go do something else.

And I find it neither surprising nor persuasive that this useless metric is standard in the "social science" field.
11.5.2007 2:05pm
Ralph Phelan (mail):
For example, his comment that measuring an agency's decision as liberal or conservative based on whether it is challenged by a public interest group is not reliable because liberal groups might challenge decisions anyway, is certainly a theoretical possibility but how likely is it?

I'd say quite likely.

Consider the hypothesis that the "permanent bureaucracy" at the EPA are mostly "liberal" on environmental issues - which is how they wound up working at the EPA rather than some other government job. I think that's plausible*. It would also be compatible with the notion that when they make a "conservative" decision it's legally bulletproof (as the only reason they made that decision is they could see no way to wiggle out of the law), while their "liberal" decisions are on dodgier ground, as those are the ones where they're going with their own desires, and are thus more likely to stretch the law.

Now assume that the permanent bureaucracy at the EPA are less "ideologically isolated" than the staffs of environmental activist groups, and thus have a better idea of how far they can actually go before being overturned. Or else assume that the risk/reward calculation for proceeding with an iffy case is different for a bureaucrat than for an activist: bureaucrats are more risk averse. In either case there'd be many instances of decisions on the EPA's part that were, "liberal, but not liberal enough" for the NGOs, and that thus got challenged from the left.

Just because someone got challenged from the left, it does not automatically follow that he is in the center.

* (So far as I know, the guys that planted Canadian lynx hair so they could protect some land are still working for the EPA. I'd count that is evidence in favor.)
11.5.2007 2:22pm