pageok
pageok
pageok
What Should a Judge's Reversal Rate Be?

http://www.concurringopinions.com/2009/05/">Dave Hoffman thinks "we have no idea what an appropriate reversal rate for an appellate judge ought to be." I am not sure I would go quite that far, but I certainly agree with him that "An appellate judge's 'reversal statistic' tells you less than you think about the 'merits' of her opinions, or even how such opinions stacked up against governing Supreme Court precedent."

[Note: Link fixed.]

Related Posts (on one page):

  1. What Should a Judge's Reversal Rate Be?
  2. Business and the Sotomayor Pick:
Steve:
Many Supreme Court decisions hinge principally on the lineup of the Court - which is why these nominations are so contentious, of course. If an appellate judge writes a careful opinion that faithfully applies the relevant precedents, it seems unproductive to score him/her as a good or bad judge based upon whether he/she correctly predicted how Justice Kennedy's vote would turn out.

What we should truly be interested in is the incidence of non-mainstream opinions, and I'm not sure that numbers can really help us much in that search.
5.27.2009 2:05pm
Tregonsee:
As someone with no legal background, I was wondering whether it makes sense to ask about the number of times, suitably normalized to the number of cases and district, a judge has decisions taken under consideration by the SCOTUS? IE, is a judge whose opinions are seldom reviewed a "better" judge one whose decisions are more frequently reviewed, even if upheld?
[Close this window]
5.27.2009 2:12pm
Pyrrho:
I tend to agree that a reversal rate really doesn't say much at all. According to Concurring Opinions blog (I can't link due to length) from 2004-2006 the Supreme Court reversed right around 75% of all cases. I'm not statistician, but I would assume that those numbers reflect the average reversal rate for any given judge (discounting, of course, judges who had no cases taken by the Court).
5.27.2009 2:12pm
Tracy Johnson (www):
Judges should charge the same rate they do to let rulings stand! :-)
5.27.2009 2:16pm
Borealis (mail):
One effect of a reversal rate evaluation is that it will rule out just about any judge in the Ninth Circuit. :-)
5.27.2009 2:17pm
Dan28 (mail):
It would be odd to have a reversal rate below 50%, since most of the time if a majority of SCOTUS agrees with the circuit opinion they don't take the case on cert.And of course, a conservative court will produce liberal judges with higher reversal rates than conservative judges.

It basically tells you nothing. We will never arrive at an OPS for judging.
5.27.2009 2:21pm
Some Dude:
The reversal rate of a super-strict, principled constructionist constitutionalist would be sky-high.
5.27.2009 2:23pm
Dave N (mail):
I agree reversal rates can be meaningless, particularly if, as Steve suggests, a judge is trying to predict how Justice Kennedy will decide.

Given the Court's caseload, I can imagine the some judges might go an entire career and have no cases reviewed by the Supreme Court.

That said, IF (and this is a big IF not related to Judge Sotomayor or any other specific judge for that matter) a judge is reversed 9-0, 8-1, or even 7-2 more than once, then there should be a red flag.
5.27.2009 2:28pm
Henry679 (mail):
It is a nonsense statistic wielded by those grasping at straws.

If somebody wants to complain about the jurisprudence of the current nominee, do it--there is certainly enough grist for the mill there. But since that will almost certainly be a losing argument, we are going to be treated to a summer of nonsense, again. This is just one minor example that has been floated--it will get dumber by the minute.
5.27.2009 2:29pm
ShelbyC:
13.7%
5.27.2009 2:29pm
GMUSL '07 Alum (mail):
I agree that it is the score of the reversals, rather than their fact, that is the more relevant concern, given the small sample size of the SCOTUS docket coupled with its discretionary nature.

A judge who, in a relatively small number of cases, is reversed 9-0 and 8-1 on the vast majority of the reversals, is, to my mind, much more troubling than a much larger number of 5-4 and 6-3 reversals.

The latter looks like somebody who, though likely to shift the court, tries to operate within precedent; the former looks like making policy from the bench.
5.27.2009 2:36pm
alkali (mail):
Given the small numbers of cases that the SCt takes relative to the output of all federal appeals courts, it is hard to imagine that any statistic would be significant. Note also that federal court of appeals judges are bound by prior precedents of the SCt and of their circuit: if the SCt decides to reverse one of its own precedents, or decides to overrule that circuit precedent, then a court of appeals judge can issue a ruling wholly consistent with binding precedent and still be reversed.
5.27.2009 2:38pm
frankcross (mail):
I've got an article with some info on this coming out in DukeLJ. For 1989-2000, the most commonly reversed opinion authors were

Reinhardt (9th) 14
Pregerson (9th) 9
Fletcher (9th) 8
Chapman (4th) 7
J. Gibson (8th) 7

The most frequently affirmed were Posner and Higginbotham, with 8 each.
5.27.2009 2:41pm
ShelbyC:
thead winner = Tracy Johnson
5.27.2009 2:45pm
Jim Rose (mail) (www):
A Supreme Court Justice's reversal rate should be zero--but then wasn't Stevens reversed yesterday?
5.27.2009 2:47pm
DiverDan (mail):
Ditto to the point made by GMUSL Alum -- I'm not certain that a reversal in a 5-4 or even 6-3 decision bothers me too much, especially on cases which hinge entirely on statutory construction. Even if a Judge is reversed 5-4 in a case like that, his analysis at least gathered a significant minority following and probably was a real good faith effort to follow the law (which I think is the most important part of any Judge's job). On the other hand, someone who is regularly reversed in 9-0, 8-1, or even 7-2 decisions (especially if it's only Breyer and Souter who agree) is probably not trying too hard to follow the law and precedent; that is likely a Judge who has gotten a god complex (an amazingly common mental defect among the Judiciary, sadly) and wants to impose his or her own view of what is "right", the law be damned.
5.27.2009 2:47pm
NaG (mail):
I agree that it is not a very useful statistic. There are too many variables affecting whether the SCOTUS hears a particular case (such as the ability/interest of the losing party to fund the appeal) to make this a useful indicator of how often a circuit judge is able to stand up to review.

What I would be more interested in is how often a judge who dissents from the panel's ruling compels the SCOTUS to agree with them. Again, this isn't a reliable indicator, but a strong dissent can sometimes compel the Justices to grant certiorari, plus it shows an independence of thought to hold out against the other two judges in the majority. I would also be interested in knowing how often the applicant judge dissents from their panel majority and then gets the en banc court to reverse the panel's ruling.

I would consider judges who rack up these "winning dissents" to be leaders in the law, at least compared to the norm.
5.27.2009 2:58pm
Allan (mail):
I guess it depends if you agree with the judge's perspective.

Take two district court judges:

1. One in the 2nd Cir. consistently rules that the second amendment protects an individual's right to bear arms. The judge is consistently reversed. But, if you agree with him, you think he is great.

2. One in the 5th Cir. consistently rules that the second amendment provides a collective right. The judge is consistently reversed. But, if you agree with him, you think he is great.
5.27.2009 3:01pm
MS (mail):
Even a 9-0 reversal can mean nothing. The Court doesn't always take the first opinion in the circuit to implicate a split. Sometimes a writing judge is just following circuit law, as she must, and then the Court grants and reverses.
5.27.2009 3:02pm
Prof. S. (mail):
I'm far less concerned about the number of times that someone was reversed as to why they were reversed. I remember a Posner antitrust opinion (case name escaping me) in which he basically said it was a terrible decision, but that he was bound by the Supreme Court. The Supreme Court then "reversed" Posner. The fact that the Supreme Court considered the judge and agreed with him by "reversing" his decision is hardly a strike against him.

I'd take a judge who is reversed 100% of the time but has a well-thought out and reasoned conclusion than one who has never been reversed beause s/he happens to play to the right partisan bend in the court.
5.27.2009 3:03pm
Notmyleg (mail):
DiverDan, any particular reason to single out 7-2 cases with Breyer and Souter (as opposed to say Scalia/Thomas) other than general agreement with Scalia and Thomas? That may sound a little snarkier than I intended, but I am curious why a repudiation by ~75 percent of the supreme court is less important depending on who sided with you.

I actually think it is very unlikely that we can get any useful information from reversal rates. First, judges are unlikely to be reviewed enough to reach any statistically significant conclusion. I have seen Sotomayor's record quoted as 3/5 reversed, or 60 percent. That's slightly better than average, but there is no way that can be considered a statistically significant sample. This will be even more limited if we go a step further, and look only at decisions with a vote of 7-2, 8-1, or 9-0.

Second, even if you had a judge with enough reviews to make a judgment, I think you would need more information. Are we talking about a case of first impression before the circuit, where the judge was interpreting the law from scratch, or is it a case where a judge is applying a past precedent. It hardly would be reasonable to punish a judge who faithfully applied circuit court precedent, only to have SCOTUS rule the other way. In fact, to ignore their circuit's precedent would be a serious problem, even if they believed it to be incorrect.

That raises a further issue. Not everyone who has a decision "overturned" by the supreme court, actually has it done in a case before the court. In the case of a circuit split on an issue, the supreme court may uphold the case it grants cert. to, thereby superceding the conflicting rulings in other circuits. The judges who decided those cases were wrong, but those cases will never show up in their reversal stats.

I think there is information to be gleaned by looking at a persons record in general, and that includes detailed looks at their reversal records. But the idea that a simple number is going to tell us much strikes me as absurd.
5.27.2009 3:06pm
one of many:
did the link change? the Hoffman piece is here.
5.27.2009 3:10pm
Terrivus:
Even 9-0 or 8-1 opinions are not always reliable indicators of the "wrongness" of the lower court's opinion; you have to look at why the Court granted cert. This is often stated in the opinion itself, but can sometimes be determined by looking at the cert petitions or questions presented.

It may be that the Court reverses a decision 9-0, but that particular decision was not the "bad" decision--it may have been an earlier one in the circuit that, for whatever reason, the Court did not take. This could be because of a vehicle problem, or there wasn't a split and it wasn't so wrong as to grant cert, or the losing party simply didn't seek cert. But when the later decision, applying that wrong precedent, comes along, a split may have developed, or a better cert petition was presented, or the Court may simply be looking to fill its docket. So it's that later decision that gets reversed.

The best indicators, I think, of the "wrongness" of a reversed opinion are the following:

1. Summary reversals. Decisions so obviously wrong the Court doesn't even need to hear argument. I am willing to bet the ranch that Judge Reinhardt is the clear leader in this category.

2. 9-0 reversals (maybe 8-1) where there was no split below. This means that the Court basically took the case because it thought the decision below was obviously incorrect. And then it reversed it by a unanimous, or nearly unanimous, vote. Again, by and large the leading circuit here is the Ninth Circuit.

2. 9-0 reversals (maybe 8-1) where there was only a 1-1 split below. This means that the Court took the case even though there was only a narrow split because it thought that one of the decisions below was likely incorrect. And then it reversed it by a unanimous, or nearly unanimous, vote. (Or, of course, it could affirm the case it took 9-0, abrogating the clearly wrong case by the same margin, casting the same mark on the other case.)

This is actually what happened to Sotomayor in the Merrill Lynch v. Dabit case.

3. Reversals of any sort where even the dissent, supporting affirmance, uses different reasoning than the panel below.

This could very well happen in the Ricci case, the panel for which Sotomayor say on and didn't really provide any reasoning in its decision.

4. Affirmances where the majority uses different reasoning than the panel below.

This also happened to Sotomayor in the Knight case.

So you can't really go by raw numbers--you have to look at the reasoning underlying the reversals. I haven't looked at enough cases to know whether Sotomayor's episodes were really all that bad, but there are enough qualitative indicators to throw up a few red flags.
5.27.2009 3:13pm
Calderon:
I'm completely confused as to Hoffman's claim that "Ironically, when a judge significantly departs from precedent either for or against the plaintiff, settlement may be particularly likely, as the parties' chances above are quite clear: thus a judge who convinces her colleagues to depart from precedent often will almost never be reviewed or reversed by the Supreme Court."

At least in my experience, settlement after a federal appellate court opinion has come down is somewhat rare. Both sides know that the chances of cert (or rehearing and rehearing en banc) are slim, and thus the winning party has little incentive to settle. Knowing this, the losing party might as well roll the dice and see what happens. Moreover, if the judge really has made a decision that departs dramatically from precedent, that gives the losing party more incentive to stick with seeking rehearing, rehearing en banc, or cert rather than giving in to the winning party.

If he's talking about settlement after you learn the panel or after oral argument, but before the decision comes down, sure that can happen. But in the case he's talking about the judges would somehow need to signal that they plan to dramatically depart from precendent during oral argument to alter the parties' perception of their chances and induce a settlement. Moreover, a single judge making such a signal during argument won't work; you'd need a least two. While I'm sure this sometimes happen, (again in my experience) it doesn't happen too often in civil cases where both sides' arguments typically have some merit. Most judges ask questions of both sides, and most counsel are experienced enough to know that questioning during oral argument isn't necessarily indicative of how a judge will vote.

Thus, I don't see how a "bad" appellate court judge could count on the parties settling to avoid review of his or her decisions. Seems like a much better argument would be the straightforward one that the sample sizes simply are too small to make meaningful comparisons.
5.27.2009 3:37pm
rosetta's stones:
And to be fair, you have to apply the Wise Old Latina Multiplier (WOLM) to those reversal statistics, in order to do a proper analysis.
5.27.2009 3:38pm
Calderon:
One other point (and I don't want to sound too partisan here) is that certainly many conservatives would argue that Reinhardt often makes decisions that depart from or are inconsistent with Supreme Court precedent. Yet as Frank Cross's statistics above show, enough of his decision remained unsettled for him to get reversed quite a few times.
5.27.2009 3:43pm
Calderon:
One last point and then I'm done. It seems like to get to Hoffman's conclusion, you need to use the exact opposite reasoning her does. Hoffman's basic claim is
"Ironically, when a judge significantly departs from precedent either for or against the plaintiff, settlement may be particularly likely, as the parties' chances above are quite clear: thus a judge who convinces her colleagues to depart from precedent often will almost never be reviewed or reversed by the Supreme Court." (emphasis added)

But settlements are not the fruit of certainty, they are the fruit of uncertainty. If I'm the winning party, and I know to a 100% certainty that the decision will be upheld, I have no incentive to settle. Thus, the unlikelihood of review by the S Ct means I won't settle.

On the flip side, a "bad" judge that departs a lot from precedent may introduce uncertainty and thus make settlement more likely. Normally, the winner in an appellate court know cert is unlikely and thus has little incentive to settle. If, however, the decision dramatically departs from precedent, the winning attorney may be more uncertain about whether it will hold up and thus more concerned about a cert or rehearing grant. Becuase of this uncertainty, the winning party may be more willing to give up some of its gains to avoid the chances of cert.

To rewrite Hoffman's conclusion, "Ironically, when a judge significantly departs from precedent either for or against the plaintiff, settlement may be particularly likely, as the poor opinion that departs significantly from precedent will introduce additional uncertainty and make the parties more likely to settle (as opposed to the normal case where the winner of an appellate decision has little incentive to settle)."
5.27.2009 3:52pm
levisbaby:
Just to interject a little reality into this thread: According to SCOTUSBLOG, "Since joining the Second Circuit in 1998, Sotomayor has authored over 150 opinions, addressing a wide range of issues, in civil cases". Even if we do not count the opinions she has authored in criminal, rather than civil, cases, that means the Supreme Court's reversal rate is not 60 percent, but at most 2 percent -- 3 cases out of 150.
5.27.2009 3:57pm
one of many:
Levisbaby

because the USSC can decide which cases it takes one can only consider reversal of Circuit Judges in cases which the USSC decides to hear. District judges are a different matter, circuit judges don;t have the discretion not to hear cases on appeal from district courts.
5.27.2009 4:16pm
Gideon's Trumpet (www):
The answer, of course, is 42 (percent).
5.27.2009 4:30pm
Assistant Village Idiot (mail) (www):
Yes, Gideon's Trumpet, how many roads must a wise old latina walk down?
5.27.2009 4:44pm
Joseph Slater (mail):
I think I get Gideon's Trumpet's reference, and I approve of it.
5.27.2009 4:45pm
Terrivus:
Since joining the Second Circuit in 1998, Sotomayor has authored over 150 opinions, addressing a wide range of issues, in civil cases. Even if we do not count the opinions she has authored in criminal, rather than civil, cases, that means the Supreme Court's reversal rate is not 60 percent, but at most 2 percent -- 3 cases out of 150.

Assuming that your 2% statistic has any validity--which it really doesn't, since the Supreme Court reviews cases at its own discretion--I'm willing to bet that 2% is actually on the high side for an appellate judge.
5.27.2009 6:58pm
frankcross (mail):
NAG, I looked at that in the article (which I don't have in front of me here). I do remember that Kozinski was very high. But I'm not sure whether that is a credit to his dissents or to the opinions of the judges with whom he happened to be sitting.

Terrivus, I think it is on the high side (if you use median not mean), but only very slightly, not with any statistical significance.
5.27.2009 7:19pm
Rich Rostrom (mail):
Slater has an excellent point - the metric could just as well be the proportion of the judge's total rulings overturned by a higher court. Another could be the judge's proportion of rulings from that circuit that are reversed.

This could be affected by factors controlling the number of cases heard by the judge. I'd guess that there are very few marathon appellate cases (unlike, say the IBM antitrust case). But some could take longer than others.

Another point might be the number of appeals from that judge granted cert; even if the Court upholds the judge, there was some issue to raise.

If a lot of certs are granted against a judge's rulings, it seems unlikely that those rulings would be usually upheld 7-2 or better; if the SCotuS decisions were often 5-4 to uphold, that says something.

BTW, how often has it happened that an appeal was granted cert and then rejected 9-0? I can see it happening where the Court wanted to make a point, but not very often.
5.27.2009 10:12pm
Joseph Slater (mail):
Rich Rostrum:

For the record, that wasn't my point.
5.28.2009 9:31am

Post as: [Register] [Log In]

Account:
Password:
Remember info?

If you have a comment about spelling, typos, or format errors, please e-mail the poster directly rather than posting a comment.

Comment Policy: We reserve the right to edit or delete comments, and in extreme cases to ban commenters, at our discretion. Comments must be relevant and civil (and, especially, free of name-calling). We think of comment threads like dinner parties at our homes. If you make the party unpleasant for us or for others, we'd rather you went elsewhere. We're happy to see a wide range of viewpoints, but we want all of them to be expressed as politely as possible.

We realize that such a comment policy can never be evenly enforced, because we can't possibly monitor every comment equally well. Hundreds of comments are posted every day here, and we don't read them all. Those we read, we read with different degrees of attention, and in different moods. We try to be fair, but we make no promises.

And remember, it's a big Internet. If you think we were mistaken in removing your post (or, in extreme cases, in removing you) -- or if you prefer a more free-for-all approach -- there are surely plenty of ways you can still get your views out.