[Ward Farnsworth (guest-blogging), August 2, 2007 at 7:26pm] Trackbacks
Puzzles About Proof.

I have a taste for conundrums and paradoxes, and The Legal Analyst discusses lots of them. Here are a couple of fun examples involving problems of proof:

1. The conjunction paradox. The standard of proof in a civil lawsuit — a case arising from a car crash, for example — is the preponderance of the evidence: the plaintiff has to prove his case by a "more likely than not" standard. So imagine a case where there are three contested issues. Maybe it's an accident case where the plaintiff has to prove (a) that the defendant was negligent, (b) that the negligence was the cause of the accident, and (c) that he has a good, truthful excuse for the fact that the claim appeared to be late under the statute of limitations. The jurors decide that they are around 60% sure that the plaintiff is right about the negligence claim, around 60% sure that he is right about causation, and around 60% sure that he is truthful in his story about the statute of limitations.

Should the plaintiff win this case? (Don't be too sure of yourself!) Would the plaintiff win this case? In other words, how do you think the jury should and would be instructed to act if it reached these conclusions?

[UPDATE. The plaintiff will probably win, because jurors generally are told to find for the plaintiff so long as they think each element of the case is proven by a preponderance of the evidence. Yet the chance that all of the elements of the plaintiff's case are true is around 22%, which seems to flunk the preponderance standard, creating not only a paradox but some potentially serious problems of policy.

There are various replies to this — that sometimes these probabilities may not be independent, or that sometimes the jury may be choosing between only two possible stories, or that we can draw additional confidence from the fact that (say) six jurors, and not just one, all reached the same conclusions. But many students of the paradox nevertheless conclude that defendants are often held liable when they shouldn't be. See, e.g., Ronald J. Allen and Sarah A. Jehl, Burdens of Persuasion in Civil Cases: Algorithms v. Explanations, 2003 Mich. St. L. Rev. 893. These ideas are discussed more in the book.]

2. Proving the law. Suppose you offer to trade a gun to a drug dealer for a couple of ounces of cocaine. The seller accepts, then announces that he is an undercover FBI agent and leads you off to jail. You are prosecuted under a statute that gives many years of prison to anyone who "uses" a firearm in relation to a drug trafficking offense.

You have two lines of defense. You plan to deny that you ever really offered the gun to the undercover agent; to overcome this denial, the government will have to prove its case beyond a reasonable doubt. So far, so good. But you also have another argument: that the prosecution has misread the statute. You don't "use" a gun if you try to barter it; you only "use" a gun (you plan to argue) by putting it to work as a weapon.

There are lively arguments to make either way on this issue (and I'm not really looking for them here). Assume that the judge thinks it's close but decides that the statute does cover your case. Well, but wait — how sure must the judge be? Is it necessary that he be convinced beyond a reasonable doubt? If not, why not? (I consider the second question — why — the more difficult and interesting part.)

[UPDATE. The puzzle is that we require a very high level of certainty when it comes to facts in criminal cases, but not when it comes to law; we are willing to award long prison sentences, or for that matter death sentences, on the basis of interpretative decisions that everyone knows may be quite doubtful. Indeed, judges do not generally confess to using any standard of proof or confidence at all when they interpret the law, with the partial, occasional, and unreliable exception of the rule of lenity (the use of which I discuss here).

The best explanation of this state of affairs, perhaps, is that if we required any particular level of confidence before a judge could state the law, there might be many situations where there ends up being no law because there is no interpretation that satisfies the standard of proof. This could have some rather untoward consequences. These ideas, again, are discussed in the book; the most interesting longer treatments, I think, are Gary Lawson, Proving the Law, 86 Nw. U. L. Rev. 859 (1992); Larry Alexander, Proving the Law: Not Proven, 86 Nw. U. L. Rev. 905 (1992).]

Christopher M (mail):
As to the first puzzle: it matters, of course, that the jurors' probability assessments of various elements will often not be independent of one another. For example, all three of the elements you mention could turn, to one degree or another, on the plaintiff's credibility when he testifies. So a juror might think there's a 60% percent chance that he's giving generally truthful testimony, and that if he is, all three elements are satisfied. In that case, the incongruity you're getting at wouldn't arise. Of course, in many cases the probabilities will be more independent than that.

Is there any empirical evidence about the relationship between jurors' (or people in general's) probability estimates of various circumstances and their probability estimates of conjunctions of those circumstances?
8.2.2007 8:45pm
Steve P. (mail):
Am I missing something? For the conjunction paradox, in your hypothetical, the plaintiff shouldn't win (since the jury is only 21.6% sure that he's telling the truth about all three things — .6^3 = .216). However, the plaintiff probably will win, simply because humans will intuitively go with the average, which is 60%. Or is this deeper than the shallow reasoning I've put forth?
8.2.2007 8:54pm
Mr. Impressive (mail):

Christopher M above addresses your question. The issue is whether the probabilities are independent. It is only mathematically proper to multiply probabilities for independent events.

The three elements here are usually not going to be independent events, since a good portion of the uncertainty will have to do with witness credibility. And witnesses credibility with respect to issue X is not usually independent to his credibility with respect to issue Y.
8.2.2007 9:09pm
Crunchy Frog:
In the second case, whether the law applies or not is a completely binary issue; either it applies, or it doesn't. There are no percentages involved.

IANAL, but it seems to me that, as opposed to a question of fact, there is no requirement that a question of law needs to be "proved beyond a reasonable doubt". As long as he (probably) thinks the law applies, he is free to apply it.

It sounds to me a great issue to raise on appeal, however.

p.s. Isn't bartering a gun (without doing to requisite background checks, etc) a crime in itself?
8.2.2007 9:13pm
As to the second puzzle, I think it pretty clear (i.e., it's not a close question for the judge) that the statute, 924(c)(1), covers bartering under the Smith and Bailey cases. But, in the spirit of the hypothetical, if it was a truely closer question of law...

I think the answer is the judge must be convinced of the legal issue by a mere perponderence of certainty.

My thoughts go like this. Everyone knows the Hand inequality B < PL for imposing a duty to make safe. The procedural due process Matthews case essentially uses the same test for LifeLibertyProperty deprivations (importance of right involved and liklihood of improper depravation of that right absent more procedure versus burden on government - yes that's BPL). The same test justifies a "beyond a reasonable doubt" fact-finder burden in criminal cases because the right taken in criminal cases is much heavier than in civil, and the liklihood of factfinder error is, if not roughly the same, a little greater in criminal cases. Setting the standard of proof higher for the factfinder in criminal cases balances both of these increased weights on the other side of the equation.

For issues of law, the same heavier right is at stake in criminal cases compared to civil cases. However, the liklihood of law-decider (judge) error is the same, if not less, than in civil cases (if anything, because criminal statues are subject to greater constitutional clarity standards). This calculus would impose a only a slightly higher burden of legal certainty on the judge in criminal cases, but not a "beyond a reasonable doubt" level of certainty. Further, because criminal defendents get more levels of law-decider review (habeus, bifurcation, etc) than civil defendants, this increased review burden serves to further balance out the equation, bringing the required level of certainty in legal issues much closer to a mere perponderence.
8.2.2007 9:20pm
Christopher M (mail):
The "rule of leniency" is not much of a real rule, since no one ever seems to follow it except when it doesn't make a difference. But in theory, it encourages judges unsure of the law in a criminal case to give the defendant the benefit of the doubt.
8.2.2007 9:27pm
David Price:
I think it's not so much a question of independent versus dependent probabilities as it is three independent *facts.* Each of those facts needs to be proven by a preponderance of the evidence, but their conjunction need not be. In the first hypo, the factfinder has found three facts by a preponderance: 1) negligence, 2) causation, and 3) valid excuse for clearing the statute of limitations. The legal conclusion that follows from those facts (assuming those are the only relevant factual determinations) is liability. As another commenter pointed out, a legal determination is not subject to a preponderance burden, since it is based on established facts rather than evidence.
8.2.2007 9:38pm
I'm not sure what's mentally taxing about the second question.
8.2.2007 10:07pm
K Parker (mail):
Crunchy Frog,
Isn't bartering a gun (without doing to requisite background checks, etc) a crime in itself?
Not in the Land of the Free and the Home of the Brave! Here, there are very few regulations on the private sale of firearms, and background checks are not among them. (And yes, I do mean to cast aspersions of un-Americanism toward Illinois and their abominable Firearm Owner ID Card, and other similar anti-freedom jurisdictions.)
8.2.2007 10:11pm
Mr. Impressive (mail):
David Price,

The issue is not merely one of facts, but rather uncertainty about the facts. Presumably, if the facts were known with 100% certainty, the case would be easy. If facts are not uncertain, then there are no probabilities whatsoever.

The question then is what causes uncertainty. Not having direct access to the facts is usually what causes uncertainty. So, we often have to rely on witnesses to determine what the facts are. But then what do you do when you have conflicting testimony? Then it is often a matter of witness credibility for reasons of memory, physical ability (eyesight, hearing, smelling etc.) or honesty.

Issues of witness credibility are not usually independent, even as to facts that are independent. A witness who is willing to lie or has a self-serving memory about X is more likely to lie or have a self-serving memory about Y. A witness with a faulty memory about X is more likely to have a faulty memory about Y. And so on.

It is usually the case that probabilities are not independent, even when the facts themselves are independent.

To add another twist. With accidents, the facts themselves are not likely to be independent. Whether a given action is negligent is likely to be highly related to whether that action is likely to cause harm to another. So, even negligence and causation, ignoring issues of witness credibility for now, are not independent. Now consider the statute of limitations issue. Certainly, a plaintiff willing to lie or who have a self-serving memory about this issue is more likely to be lying or have a self-serving memory about the circumstances surrounding an accident and their descriptions of the defendants actions.

So, to answer the question proposed by Mr. Farnsworth, should the plaintiff win their case? The answer is a definite maybe. It depends on the degree to which our uncertainty is based on independent versus dependent factors.

What do jurors do in real life? Well, for one, jurors in real life do not understand probability theory. So, obviously, they do not consider these issues.

Does that matter? Probably not. All the probabilities that Mr. Farnsworth uses are completely made up. They are purely subjective probabilities that are not subject to empirical verification. You can not run trials with a particular witnesses credibility to determine that they lie or have a self-serving memory approximately 60% of the time. Certainly, such probabilities, even if they were empirically possible to produce, would not be avialable to a jury in a trial. The only way to get such numbers would be to pull them out of your ass (excuse the expression). That jurors would not properly assess made up probabilities is not a big concern to me. Especially since, for the reasons I have noted above, even an expert in probability theory would not be able to decide the case based on the limited information Mr. Farnsworth provided, even if the probabilities were available and 100% certain to be accurate.
8.2.2007 10:40pm
Daniel San:
1. Start with C. Without it, you don't get in the door. Once you are in the door with > 50%, it shouldn't factor in the decision. A and B can be combined into a single question: Did D negligently cause the accident. Serious doubts (40%) about whether D was negligent and serious doubts (40%) whether that negligence caused the accident, a jury likely (and should) find for D.

2. The prosecution must prove the elements of the offense beyond a reasonable doubt. It need not prove what those elements are.
8.2.2007 11:37pm
BruceM (mail) (www):
Christopher M: It's the rule of lenity, not rule of leniency. And it is only employed when a criminal/penal statute is vague or ambiguous (i.e. give the defendant the benefit of the doubt as to the multiple interpretations).

With modern summary judgment practice, it would be a pretty rare case where the issue of statute of limitations is submitted to the jury. Maybe something having to do with the discovery rule, i.e. when the plaintiff knew or should have known her cause of action had accrued. But even then, it would probably be disposed of one way or another via summary judgment.
8.3.2007 12:13am
Steve P. (mail):
Mr. Impressive —

I had assumed, based on the phrasing of the question, that the facts were objectively independent (but that the jurors might not recognize that). Your reasoning is right, of course; if there is an X factor (presumably plaintiff credibility), it could most certainly indicate that the three questions were related, and thus change the nature of the result.
8.3.2007 12:29am
I'm with anonVCfan: I don't understand what the puzzle is with the second problem. Standards of proof always vary tremendously depending on the context. There are different standards for facts, law, evidentiary rulings, appeals, etc.

Usually those different standards reflect different distributions of power for different types of issues: We might be comfortable giving a court the power to say what the law is generally but feel less comfortable giving the court the power to say what the facts are in a specific case.
8.3.2007 12:55am
Jay Myers:
In the first question, which bears no relation to a paradox, the epistemic duty of the jurors is to decide the truth of each element independently and then choose the verdict that is sufficiently justified by proven elements. Thus, they should find for the plaintiff.

Psychologically, however, people are hardwired with several logically invalid tendencies. One of those tendencies would result in jurors wrongly considering the probability of the case as a whole much as they would wrongly try to consider the probability of four coin flips all resulting in heads. "The plaintiff's argument about A sounds plausible as does his case for B and C but there is a fair degree of likelihood that each of them are false so, really, what are the odds of A, B, and C *all* being true?"

For the second case, I would think that the "reasonable doubt" requirement only applies to the defendant's guilt as a matter of fact and not to the judge's (or jury's, since the jury is the ultimate arbiter of both fact and law) belief in the interpretation of the law that they are using. Certainly they must believe in the truth of the interpretation they are using but don't need such a degree of certainty as to believe that there are no other possible reasonable interpretations of the law in question. Thus the judge and jury only need justified belief in the interpretation of the law that they are applying.
8.3.2007 1:58am
These questions really remind me of the squirrel paradox, posed and solved by the original pragmatist, William James.

1. David Price has a plausible explanation--his understanding of the question is that it is *really* asking the jury for three separate findings. The jury should find A if it believes P(A)>50%, it should find B if it believes P(B)>50%, and it should find C if it believes P(C)>50%; thus, the plaintiff prevails. Others argue that the jury must decide whether the probability of all three events occurring simultaneously is > 50%, making the victor less clear.

Why should we turn this into a metaphysical argument about whether Farnsworth's question *really* resolves to one horn of the dilemma or the other, though? This is only paradoxical because the original question is ambiguous; the solution is for the questioner to be more clear about what he means, not to debate what is *really* meant by the ambiguous question.

The only other way to resolve it is by making historical arguments or policy arguments, but that's never a satisfying way to resolve hypothetical logical paradoxes. (However, I think this is how we should solve ambiguities like this in practice.)

2. I would make the same point about #2. Does the heavy burden of proof apply only to the fact-finding or to interpretations of the relevant statutes as well?

Well, by the terms of the question, it's not clear. I suspect this ambiguity can only be resolved with reference to policy arguments or historical arguments; we're not going to get very far if we argue about what "proof beyond a reasonable doubt" *really* means as a metaphysical matter.

I suspect the point of this exercise is to point out that some questions which appear to be paradoxes aren't actually paradoxes, but are just ambiguous. Resolve the ambiguity by being more precise in defining terms. Or, when it's too late to change the terms, decide based on history or policy preferences, and then stick to that decision later.
8.3.2007 3:28am
Christopher M (mail):
BruceM: Yeah, you're right that it's generally called the rule of lenity. ("Leniency" does appear in a couple court opinions, but you're right, I just forgot the vastly more common term.) But I don't understand your next comment. Yes, it only applies when a statute is ambiguous, but in the hypothetical here, the judge is not entirely sure what the statute means -- isn't that the definition of ambiguity?

AnonVCfan and Prof. Kerr: The issue in the second question is this: We require proof of the facts beyond a reasonable doubt because we think it's really important not to wrongfully convict someone. But if we think that, then why would we allow someone to be convicted when it's less than entirely clear that the criminal statute covers their actions?

There are plausible answers, but Prof. Kerr's doesn't seem adequate. Distribution of power between judge and jury is not the issue here: whether the factfinder is a judge or jury, we impose a "reasonable doubt" standard on the facts but only a preponderance standard on the law (the rule of lenity aside). Why? Either way, we're not sure that the defendant has actually committed a crime.
8.3.2007 4:07am
markm (mail):
Christopher: I suspect the real-world answer is that the courts are run by lawyers who've been elevated to judges, and thus they are often run for the benefit of lawyers.

There's a much larger paradox than the difference in the burden of proof between questions of law and questions of fact in criminal cases. "Ignorance of the law is no excuse", even when it's impossible to know the law because experts in the field can argue endlessly about what it means, and the law is not settled until someone has allegedly violated it and the question is litigated all the way to the Supreme Court - and then, if the justices go 5-4 against the defendant, he goes to prison. The defendant may be a middle-school dropout with an IQ of 80, but he's held responsible for understanding a law that the best legal minds disagree about.
8.3.2007 9:07am
Although I disagree that either question presents a true paradox, I appreciate Farnsworth's posting an update to tie together all the comments. Would that more conspirators did likewise.
8.3.2007 9:50am
Christopher M writes:
There are plausible answers, but Prof. Kerr's doesn't seem adequate. Distribution of power between judge and jury is not the issue here: whether the factfinder is a judge or jury, we impose a "reasonable doubt" standard on the facts but only a preponderance standard on the law (the rule of lenity aside). Why? Either way, we're not sure that the defendant has actually committed a crime.
Christopher, I never said or suggested that the issue was distribution of power between a judge and jury: I said the issue was distribution of power. Consider: Proof beyond a reasonable doubt is *not* required in about 95% of criminal cases. These cases are resolved through a plea agreement. Why the different standard? I don't think it's a puzzle or a paradox; it's just a distribution of power to ensure a practical and efficient system given the likely consequences of different standards of proof at different stages.
8.3.2007 10:51am
Damon Katz (mail):
Something is not clear to me:

In problem 1, the jury is asked to decide a statute of limitations issue. Why is that for the jury? Isn't the limitations defense a matter of law, for the judge?

Thanks. Sorry if this is obvious to the other lawyers. In my defense, I'm a crim. pro. guy.
8.3.2007 11:05am
Sk (mail):
I have a more systemic question that I have always wondered about in this type of situation. and that is, what does 'preponderance of evidence,' and similar concepts like '60% likelihood of being correct' mean? If I were on a jury, and I was given this type of instruction, I would have no idea what to do.

for instance: given the evidence presented in, say a car accident case, I could imagine thinking the driver was at fault, thinking the driver wasn't at fault, and not being sure. I really have no sense of how to consider whether, either the driver was 60% at fault, or that I am 60% confident that the driver was at fault. It simply makes no sense to me. It seems to me the whole 'preponderance' or '60%' test is really a scientific-sounding means of saying 'if you aren't sure who is at fault, what does your gut say?' (which, in my mind, shouldn't be legally defensible).

I realize this question may be more philosophical than you intend, but every time I hear of a decisionmaking standard in a civil case (this example: the OJ Simpson civil suit is another one), I really can't get past my discomfort with the concept to wonder about other issues.

8.3.2007 11:24am
Ward Farnsworth (mail):
Dear all -- Sorry I haven't gotten my act together to reply to comments earlier in the week. The recent one above from "sk" provokes a familiar reaction for me, though -- good point; it's discussed in the book, but I didn't feel that I had space to get into that side of the problem here. On my first day there was a comment complaining that my post was too long; on other days, complaints that information or nuances or qualifications were omitted. My conclusion: it's hard to adapt book chapters to the blog format!

In this particular case, I really didn't mean to start a debate about whether either of the issues I've raised is an authentic "paradox." Please pardon my casual usage of the word; I can't see the point in fussing much over it, and "conjunction paradox" is a standard term in the literature. I'm really more interested in the substantive question of why we do things the way we do.

I see an interesting puzzle to think about in the discrepancy between the high confidence we require in factual judgments in criminal cases and the low confidence we require for the legal variety. Orin, who I admire, does not; this may just suggest that he's less easily amused than I am. Vive la difference! If you like thinking about that sort of thing, the book should interest you; if not, perhaps not. But check out the sample chapters to get a better sense of this.

In any event, I do appreciate the time many of you took to speculate or comment on the questions I raised this week. I hope the book will provide fun reading for some of you. And no (for the benefit of whoever kept asking), I'm no relation to Allan Farnsworth. Best, WF
8.3.2007 11:38am
theobromophile (www):
The problem with compounding the probabilities in No. 1 is that it is significantly more difficult to win a case with more elements (or more facts which need to be proved).

If the plaintiff had to prove two elements, he would win if the jury were 71% sure of each element; three elements, 80% sure; four elements, 85% certain; and five elements, 88% certain.

You quickly arrive at the point where a plaintiff would need to prove each element or each fact by clear and convincing evidence (maybe 80ish% certain?) to win by a preponderance of the evidence. The biggest variable in a trial would be the number of elements in the claim at issue.
8.3.2007 10:18pm
Jay Myers:

The puzzle is that we require a very high level of certainty when it comes to facts in criminal cases, but not when it comes to law; we are willing to award long prison sentences, or for that matter death sentences, on the basis of interpretative decisions that everyone knows may be quite doubtful.

It's not much of a puzzle. We have a higher standard for that which is capable of being known with a much greater degree of certainty.

On the one hand we have facts, which by definition posses the potential for being known with a high (but not absolute) degree of certainty. On the other hand we have texts, which are open to a near-infinite number of differing interpretations. We can have opinions, sometimes very strong ones, regarding the relative merits of interpretations but it is impossible to 'know' which interpretation of a text is right. By many theories of language there simply is no single correct interpretation.
8.4.2007 1:11am