The Volokh Conspiracy

Thirty years after Merrell Dow Pharmaceuticals pulled the morning sickness drug Bendectin off the American market in response to a waive of junk science-inspired claims that it caused limb reduction birth defects, the product is returning to the market under a new name with a new manufacturer.  Contrary to what one might reasonably surmise from the linked report, the FDA never pulled Bendectin from the market; rather, no one was willing to market the drug given the litigation climate in the United States.  American women who wanted the drug, which still has no substitute, have had to either make a homemade concoction from its ingredients, as some doctors recommended, or get it from Canada.  This has meant, in practice, that few American women have used it.  The result has been much needless suffering, including a hospitalization rate for morning sickness double in the U.S. compared to Canada. Ironically, the absence of Bendectin may have even led to an increase in the rate of birth defects because some women with morning sickness are unable to keep food down and give their babies proper nutrition.  I discuss the Bendectin litigation and the negative health ramifications it caused in some detail in an article in the Michigan Law Review.

The only positive aspect of the Bendectin litigation is that the contrast between jury verdicts for plaintiffs and the overwhelming contrary scientific evidence was such that it led several federal courts to issue decisions radically (for the time) limiting plaintiffs” ability to proffer dubious causation evidence, which in turn led to the Supreme Court’s “Daubert Trilogy” and amended Federal Rule of Evidence 702.  By tightening the rules for the admissibility of expert testimony, these changes have made a recurrence of the tragedy of the Bendectin litigation unlikely.

In related news, I was just reading that a product that David Kessler’s FDA did pull off the market for political reasons with no scientific basis for doing so, silicone breast implants, have made a huge comeback, and are now used in hundreds of thousands of surgeries in the United States annually.  Unfortunately, the judiciary’s crackdown on dubious science didn’t come quickly enough to save the manufacturers from paying out billions of dollars in jury verdicts and settlements.

Economist David Friedman has an insightful post on the problems inherent in deferring to the views of “authoritative” scientific bodies:

A pattern I have observed in a variety of public controversies is the attempt to establish some sort of official scientific truth, as proclaimed by a suitable authority—a committee of the National Academy of Science, the Center for Disease Control, or the equivalent. It is, in my view, a mistake, one based on a fundamental misunderstanding of how science works. Truth is not established by an authoritative committee but by a decentralized process which (sometimes) results in everyone or almost everyone in the field agreeing.

Part of the problem with that approach is that, the more often it is followed, the less well it will work....

The first time it might work, although even then there is the risk that the committee established to give judgement will end up dominated not by the most expert but by the most partisan. But the more times the process is repeated, the greater the incentive of people who want their views to get authoritative support to get themselves or their friends positions of influence within the organization, to keep those they disapprove of out of such positions, and so to divert it from its original purpose to becoming a rubber stamp for their views. The result is to subvert both the organization and the scientific enterprise, especially if support by official truth becomes an important determinant of research funding.

I. The Dangers of Deference to Biased Experts.

Friedman makes two important points here. Scientific truth cannot be established by the endorsement of an authoritative body such as the NAS or the CDC. And if people start to take the pronouncements of such expert bodies as gospel, there is an obvious potential for abuse.

Both problems are exacerbated in cases where the scientific question at issue is relevant to some hot-button political controversy. When it comes to politics, most people have strong incentives to be “rationally ignorant,” and therefore devote little time and effort to determining whether the pronouncements of “experts” are really backed by evidence or not. Given the very low chance that your vote in an election will be decisive, there is little incentive to make a serious effort to double-check the pronouncements of experts on political issues, if your only motivation for doing so is to figure out which candidate or party has the “right” position on a given issue. For similar reasons, voters tend to be highly biased in evaluating whatever information they do learn about politics, often acting as “fans” for their respective party or ideology rather than as objective truth-seekers. This often leads them to place excessive credence in real or imagined experts who support their preexisting views, while discounting those on the other side.

II. Why Deference is Often Unavoidable.

That said, I don’t believe we can simply dispense with deference to scientific experts. There are so many complex issues in the world that none of us have the time or expertise to really delve into the evidence on more than a small fraction of them. As I explained in reference to the “Climategate” controversy in 2009:

[O]ur knowledge of complex issues we don’t have personal expertise on is largely based on social validation. For example, I think that Einsteinian physics is generally more correct than Newtonian physics, even though I know very little about either. Why? Because that’s the overwhelming consensus of professional physicists, and I have no reason to believe that their conclusions should be discounted as biased or otherwise driven by considerations other than truth-seeking. My views of climate science were (and are) based on similar considerations. I thought that global warming was probably a genuine and serious problem because that is what the overwhelming majority of relevant scientists seem to believe, and I generally didn’t doubt their objectivity.

Even if you consider yourself a great skeptic, I suspect that you too defer to expertise on many issues. You probably follow your doctor’s advice on what medicine to take when you are sick, usually without first reading up on the scientific literature on that medicine’s effectiveness, and almost certainly without performing your own laboratory experiments to assess its potency first-hand.

III. Increasing Our Expertise on When to Defer to Experts.

Given the near-inevitability of deference to experts, can we avoid the pitfalls Friedman rightly emphasizes? There’s no perfect solution. But some rules of thumb can help. First, deference to expertise is more warranted in cases where there is an expert consensus that crosses ideological lines. Like the rest of us, experts are prone to ideological bias. Thus, if experts of differing ideologies converge on the same conclusion, that’s a sign that the resulting opinion is really driven by expertise rather than bias. It doesn’t prove that the experts are right, of course, but it does justify a stronger presumption in their favor. When, on the other hand, experts do split along ideological lines, that suggests the issue is more disputable, and that bias may be influencing their judgment. It doesn’t mean that the experts are wrong or that their expertise is useless. Their views are still probably worth listening to more than those of laypeople. But it does mean that we should be more cautious about concluding that an expert pronouncement must be correct simply because the person or the institution making it has impressive credentials.

A weaker but still significant indicator of expert reliability is to ask whether expertise makes you more likely to support a given conclusion, after controlling for ideology and other factors that might bias judgment. For example, if experts in a given field are 50% more likely to believe X about a key controversy in their area of expertise than are otherwise comparable non-experts, that is some indication that X derives some support from the evidence and relevant expert analysis thereof. Bryan Caplan’s research on the differences between economists and laypeople on economic policy issues is a good example of this kind of analysis. He shows many issues where expertise in economics has a major effect on policy views even after controlling for ideology, self-interest, and various relevant demographic variables. That doesn’t mean that economists are necessarily right about those economic issues where they differ from laypeople. But it does suggest that the difference really is a product of their expertise and is therefore entitled to greater deference than a supposedly expert judgment that is mostly driven by ideology or narrow self-interest.

Finally, as in the Climategate controversy, it may be worth considering whether experts in a given field have good incentives to pursue the truth, or whether theose incentives are skewed by funding sources or by the ability of one faction to “freeze out” those who dispute the received orthodoxy. However, crude analysis of funding incentives can be even more misleading than simply ignoring them entirely. Unfortunately, properly assessing the impact of incentives on the range of views expressed by experts in a given field itself often requires detailed knowledge that most of us do not have.

Such rules of thumb don’t matter much in cases where you know enough about the field in question to assess the evidence for yourself. But in the many situations where we must defer to experts, they might help reduce the dangers inherent in doing so.

What’s wrong with this picture? (Or this site?)

I generally have a laissez-faire attitude toward product claims, so long as they are arguably true. If a claim is literally false, however, I think that’s a problem.
(Ht: Hanah Volokh)

In November, Californians will vote on Proposition 37, a ballot initiative to impose a mandatory labeling requirement on all foods produced with or from genetically modified organisms (GMOs). For reasons I discuss in this New Atlantis article, this requirement is unnecessary, unwise and potentially unconstitutional.

The effort has been endorsed by numerous progressive organizations and the California Democratic Party. Of note, those who usually police the misuse or politicization of science have been strangely quiet about the misleading and inaccurate scientific claims made by Prop. 37 proponents. Although the proposition warns of “adverse health consequences” from genetic engineering of foods, there is not a single documented case of adverse health consequences due to the use of GMOs. Yet about traditional crop-breeding techniques, we can say no such thing. It’s no wonder that the National Academy of Sciences has issued numerous reports concluding that the use of modern genetic modification techniques, in themselves, have no bearing on the relative safety of a food product. What was done to a specific GMO matters more than whether specific modification techniques were used.

It is even misleading to single out crops and other organisms modified by modern genetic modification techniques as “genetically engineered. Many common crops are “genetically engineered” in that they are the result of direct human modification. Corn, for example, does not exist naturally. It was “engineered” by humans, albeit using less precise breeding methods centuries ago.

The organizers of the effort claim consumers have a “right to know” whether their foods contain GMOs. But nothing stops consumers from obtaining such information. Organic producers and others who wish to cater to those who dislike GMOs are free to label their products accordingly (and, in my view, should be able to do so without some of the excessive disclaimers urged by the FDA). Absent evidence of a potential health risk, there is no reason for the government to mandate GMO labels. Such labels are not necessary to protect consumers against misleading claims, and a proclaimed “right to know” does not constitute a substantial governmental interest.

Some consumers may want to know whether products contain GMOs, just as others may wish to know whether a product was made with union labor, a company’s executives donated to particular political candidates, or its products were blessed by shaman priestesses. Yet it must take more to justify compelling speech in the form of product labels. Were it otherwise, there is no end to what could be the subject of mandatory labeling requirements, and there would be no meaningful constitutional protection of compelled commercial speech.

Most existing labeling requirements can be justified on the grounds that they protect uninformed consumers from potential adverse impacts. Ingredient labels, for example, protect those with allergies or specific dietary needs. GMO labels, on the other hand, do no such thing. Rather they stigmatize products, suggesting there is something significant, or even potentially wrong, with a product that was produced in this way, even there is no scientific basis for making such a claim. Some consumers may have moral or other objections to GMO products, and that is their right. Such consumers are free to seek out producers who will make products in accord with their preferences. But GMO opponents should not have the right to force others to modify product labels, at their own expense, just to satisfy one group’s set of subjective value preferences.

Does this mean there will be no GMO labels? Not at all. There is no requirement that producers identify whether products are “organic” or “kosher,” and yet such labels proliferate. Where such information is likely to influence consumer behavior, producers have ample incentives to provide the information consumers want. That is, those producers whose products are GMO-free have every incentive to disclose, and perhaps even advertise, this fact. Such disclosure is sufficient to let those consumers who oppose GMOs shop accordingly without imposing the cost of such preferences on others.

Earlier this week, the Supreme Court issued a fractured opinion in Williams v. Illinois, a Confrontation Clause case. The bad news (for just about everyone) is that this opinion hopelessly muddles things. The good news (for me and my co-authors) is that four different Justices cited the treatise I co-authored with Penn State’s David Kaye and UCLA’s Jennifer Mnookin, The New Wigmore: Expert Evidence. The further bad news is that we agree with the dissent.

I use “we”, but authorship credit for the relevant section goes to Jennifer, and editing credit to David. In other words, I had nothing to do with it.

I am, however, hoping that Williams will get the treatise some attention. It can’t hurt that Justice Kagan called The New Wigmore “the principal modern treatise on evidence.”

I’d provide an Amazon link for the volume but there isn’t one for the second edition (marketing the volume beyond library sales has not been the publisher’s strength). It is, however, possible to buy the volume directly from the publisher.

There is a lot of neat and useful material in the volume, including, e.g., a uniquely detailed discussion of the admissibility of so-called learned treatises, a topic to which other treatises devote at best a few paragraphs. Also, unlike other evidence treatises that either focus on either federal law or specific state rules, we cover the evidence law across the U.S. As a result, we extensively discuss state rules such as the Frye general acceptance test for the admissibility of expert testimony, which remains the law in many populous states including California, New York, and Florida.

When my younger daughter was getting some of her first vaccinations, I asked our pediatrician how he would feel if we refused to have her vaccinated. His reply: “I’d ask you to find another doctor.” This point of view appears to be spreading. As the WSJ reports there is an apparent increase in the number of doctors “firing” patients who refuse vaccinations.

Medical associations don’t recommend such patient bans, but the practice appears to be growing, according to vaccine researchers.

In a study of Connecticut pediatricians published last year, some 30% of 133 doctors said they had asked a family to leave their practice for vaccine refusal, and a recent survey of 909 Midwestern pediatricians found that 21% reported discharging families for the same reason.

By comparison, in 2001 and 2006 about 6% of physicians said they “routinely” stopped working with families due to parents’ continued vaccine refusal and 16% “sometimes” dismissed them, according to surveys conducted then by the American Academy of Pediatrics.

Vaccination rates have declined in recent years, largely due to completely unfounded fears that vaccines cause autism or other problems. Non-immunized children aren’t the only ones at risk when vaccines are refused. Lower vaccination rates make disease outbreaks more likely as herd immunity is compromised. Many experts suspect recent outbreaks of measles and whooping cough are the result of declines in vaccination. Perhaps if more doctors insist on vaccinations as a condition for care, more parents will get the message.

Times:

In his last years, Steven P. Jobs veered from exotic diets to cutting-edge treatments as he fought the cancer that ultimately took his life, according to a new biography to be published on Monday. His early decision to put off surgery and rely instead on fruit juices, acupuncture, herbal remedies and other treatments — some of which he found on the Internet — infuriated and distressed his family, friends and physicians, the book says.

Really? A man on the cutting edge of technology like Jobs eschewed scientifically proven treatments (at least initially) for new age nonsense? How disappointing, and sad for him and his family.

Going on the offensive against Texas Governor Rick Perry for issuing an executive mandate that young girls receive a vaccine against HPV, Rep. Michele Bachman embraced the fringe (and thoroughly discredited) claim that vaccination can cause mental retardation.  Details here and here.  It is understandable that a parent whose child experiences difficulties will be distraught and search for answers, but to give credence to the claim that vaccination causes mental retardation, autism, or other disabilities is thoroughly irresponsible.  It is one thing to debate whether a state government should mandate that children are vaccinated against something like HPV, and whether a voluntary opt-out provision is protective enough of parental prerogatives.  It is quite another to suggest that mandated vaccination creates serious health risks when there is no evidence to support such a claim.

For what it’s worth, I criticized Senator John McCain for a similar offense in 2008.

UPDATE: Henry Miller reports some additional things Rep. Bachmann should know about the HPV vaccine.  Even if Gov. Perry was wrong to order the vaccinations, there’s absolutely no basis for suggesting the vaccine is a threat to children.

I foolishly managed to once again entangle myself in a debate with Brian Leiter. There is at least one good reason not to try to engage in a reasoned blog debate with Leiter, which is that he doesn’t believe in it:

I am sometimes presented with the following criticism: “Your rhetorical style won’t persuade anyone who doesn’t already agree with you.” That is no doubt true, but, as we’ve just remarked, it is quite rare to persuade anyone by a careful, reasoned argument–indeed, so rare, that I don’t see it as worth the effort to try to do so on a blog....

Nevertheless, since I started it, I suppose I should respond. (And at least I got a laugh out of being called an ideologue by the to-the-left-of-Noam-Chomsky Brian Leiter.)

The issue at controversy is my original claim in a brief comment to this blog post on the pseudo-scientific nature of Marxism, that Freud’s “work (or at least the vast majority of it) can’t stand up to the scientific method,” to which Leiter responded that my claim is “wholly false.”

It’s rather well-established that Freud’s work generally didn’t follow the scientific method, e.g., Freud did not reach his conclusions via testing, replication, and other indicia of scientific inquiry. Moreover, Freud’s followers for decades argued that his work shouldn’t be subject to empirical testing. One could argue that Freud’s theories were still better than ones preceded it (though my understanding is that the triumph of Freudian theory extinguished some other promising lines of research), but the hostility of Freudians to scientific methodologies then retarded further progress in psychiatry for decades.

Leiter instead argues that some of Freud’s “theory of the mind” has recently been empirically validated by OTHERS who did use the scientific method. If all Leiter is arguing that not everything Freud wrote turned out to be false, he won’t get any argument from me [though that's not the same as showing the most, or even much, of Freud's work has been empirically validated. On reflection, any such debate would involve some difficult definitional boundaries: what level of generality are we talking about (e.g., who would dispute that the pursuit of sex is a important motivating factor in human affairs?); how much weight do you give to different aspects of Freud's work to define "much"?; What if a particular conclusion was correct, but the rationale was wrong? The one article Leiter cites in support of the scientific validity of Freudian theory actually acknowledges, in the abstract no less, that while modern Freudians build on Freud's genuine insights, Freud's "version of psychodynamic theory" is "archaic," and that most Freudian clinicians consider it "obsolete".]

But if Leiter is arguing that Freud’s work was itself scientific that’s another story. There’s no contradiction between having great insights into human nature–great philosophers, authors, religious thinkers, etc., have had them–and not following, or purporting to follow, the scientific method.

For some bizarre and unexplained reason, Leiter seems to think that my beef with Freud is ideological, as if acknowledging what he calls the “scientific status” of Freud’s theories would somehow conflict with my belief in ... what exactly? Perhaps he should instead note that I’ve been writing about junk science–left, right, and (mostly) otherwise–for over twenty years.

One might also give some thought to the many homosexuals, schizophrenics, victims of sexual abuse, and others who sought counsel from Freudian analysts, only to be fed nonsense about their mothers, accused of fantasizing, and so on. Leiter hasn’t acknowledged this, but perhaps he could withhold some of his vitriol from me, and spare some sympathy for these victims of pseudo-science–assuming, of course, that he agrees that they were such.

OK, that’s a bit of an exaggeration of what the district judge said, but not much. Prof. Nita Farahany (Law & Biosciences Digest) has the scoop, and a link to the decision (which came in late January). An excerpt from Prof. Farahany’s post:

The Defendant in this case, convicted of possession of child pornography, was given an “unreasonable” sentence based on the judge’s invented genetic theory that the defendant’s incorrigible genes made him act the way he did, and that there was nothing he could do about it.... [The Second Circuit] took the extraordinary measure of vacating the sentence and remanding the case to a different judge.

Note that the Second Circuit expressly said that the judge didn’t have to trust psychologists if he didn’t want to, and could look at the evidence that this defendant was indeed likely to repeat his offenses (there was some in the record). Nor did the Circuit hold that real evidence of a defendant’s genetic predisposition couldn’t be used (an interesting question on which I don’t think there’s a settled answer). But the court rightly said that speculation about what genetics will find 50 years from now can’t be part of the decision.

The NYT summarizes a new report in Science that finds a majority of high school biology teachers skimp on their teaching of evolution to avoid controversy and a sizable percentage explicitly teach creationism.  One of the study’s authors believes the answer is more and better science education for teachers.

“Students are being cheated out of a rich science education,” said Dr. [Eric] Plutzer, a professor of political science at Penn State University. “We think the ‘cautious 60 percent’ represent a group of educators who, if they were better trained in science in general and in evolution in particular, would be more confident in their ability to explain controversial topics to their students, to parents, and to school board members.”

Perhaps, especially if this education helps teachers explain that a belief in evolution does not require rejecting religious understandings of the world, let alone atheism. Creationism and “intelligent design” aren’t science, and it’s wrong to present them as such. But evolutionary theory is not a comprehensive explanation of human existence and doesn’t disprove the existence of God. Science educators would have an easier time teaching evolution if some of evolution’s advocates were not so strident in suggesting that belief in evolution disproves a belief in God. A frontal assault on someone’s worldview is not the best way to get them to listen.

This recent TV performance by Rep. Jack Kingston (R-GA) is quite embarrassing. The worst part is, I doubt he’s embarrassed by it.

Salon has withdrawn a 2005 article by Robert F. Kennedy Jr. (aka “America’s Most Irresponsible Public Figure®“) alleging a link between vaccines and autism.  The article, co-published with Rolling Stone, had an “explosive premise,” according to Salon‘s editors.  Too bad it was scientifically bogus and completely irresponsible.  Salon was forced to publish several corrections to the article, but has now decided to remove it from their website altogether.  A better step would have been to leave it online with a bolded header explaining the article has been thoroughly discredited and never should have been published in the first place.

Last year, The Lancet withdrew a controversial study that had purported to find a link between autism and childhood vaccination because “several findings” of the paper were “incorrect.”  The study, by Wakefield, et al., was the only study published in a respected medical journal reporting a potential link and was routinely cited by those claiming childhood vaccinations could cause autism, despite obvious flaws and a wealth of contrary research.  Now a new report shows the study was not just wrong, it was “fraudulent” as well.

A report by journalist Brian Deer in the British Journal of Medicine, the first in a series, reveals that the Wakefield study relied upon “bogus data” that was “manufactured” by those who conducted the study.  Specifically, Deer found that the study’s authors misrepresented medical and other information about the children in the study, including the timing and appearance of relevant symptoms, creating a false impression of a vaccine-autism link that was not there.

An accompanying editorial in the BMJ pulls no punches.

The Office of Research Integrity in the United States defines fraud as fabrication, falsification, or plagiarism. Deer unearthed clear evidence of falsification. He found that not one of the 12 cases reported in the 1998 Lancet paper was free of misrepresentation or undisclosed alteration, and that in no single case could the medical records be fully reconciled with the descriptions, diagnoses, or histories published in the journal.

Who perpetrated this fraud? There is no doubt that it was Wakefield. Is it possible that he was wrong, but not dishonest: that he was so incompetent that he was unable to fairly describe the project, or to report even one of the 12 children’s cases accurately? No. A great deal of thought and effort must have gone into drafting the paper to achieve the results he wanted: the discrepancies all led in one direction; misreporting was gross. Moreover, although the scale of the [General Medical Council's] 217 day hearing precluded additional charges focused directly on the fraud, the panel found him guilty of dishonesty concerning the study’s admissions criteria, its funding by the Legal Aid Board, and his statements about it afterwards. . . .

Meanwhile the damage to public health continues, fuelled by unbalanced media reporting and an ineffective response from government, researchers, journals, and the medical profession. Although vaccination rates in the United Kingdom have recovered slightly from their 80% low in 2003-4, they are still below the 95% level recommended by the World Health Organization to ensure herd immunity. In 2008, for the first time in 14 years, measles was declared endemic in England and Wales. Hundreds of thousands of children in the UK are currently unprotected as a result of the scare, and the battle to restore parents’ trust in the vaccine is ongoing.

(citations omitted)

Perhaps now, finally, the vaccine-autism charade is over. I’ll await the reports on Oprah and MSNBC’s “Countdown.”

More here.

Today the U.S. Court of Appeals for the Federal Circuit affirmed the Special Master’s finding that plaintiffs had failed to demonstrate any link between childhood vaccines and autism or autism spectrum disorders.  More from BLT here.

I’ve previously blogged on the efforts to link autism and vaccines here, here, here, and here.