pageok
pageok
pageok
Freakonomics and the Car Seat Debate:

I'm rapidly becoming addicted to the Freakonomics blog. The latest brouhaha on child safety seats, airbags vs. seat belts, etc., is quite interesting. Levitt argues that the data fails to support the hypothesis that car seats are safer than seat belts for children over the age of 2 years who are involved in an accident. He was criticized by a letter in the New York Times as being "irresponsible and dangerous" for publishing this conclusion and the fear was expressed by the letter-writers that they hope this "misleading article does not cost a child his life."

Also, see Alex Tabarrok's take on the child seat debate, criticizing the subordination of the scientific issue to activist conclusions.

Mike Rappoport sees something similar in legal scholarship, but I think there is one difference--I think that law professors may be more comfortable with intellectual disagreement (having been weaned nursed on the adversary system) and so that in theory there may be more of a check on the potential negative efffect of "activist scholarship" than perhaps for medicine/public health. I have the utmost respect, admiration, and deference to the medical profession, but would I be rocking the boat too much to raise the possibility that those from that world sometimes can be, shall we say, perhaps a bit paternalistic and a bit impatient with those who question their authority? I remember one medical person telling me that the very idea that there could be such thing as "lower-risk tobacco" (such as chewing tobacco vs. cigarettes) was like saying that someone could be "a little pregnant." (I reminded this person that at the very least she had to admit that the second-hand smoke risks associated with chewing tobacco are somewhat lower than for cigarettes.) The idea that Levitt's scientific conclusions from the data on car seats could be "irresponsible and dangerous," as opposed to perhaps "incorrect" or "unsupported," is really sort of a peculiar and unhelpful way to think about what is, after all, a positive empirical question, not a normative question.

Not to mention that the letter's kicker, "We hope that this misleading article does not cost a child his life," completely misconceives the public policy question. Again, I don't know if Levitt is correct. But if he is, then this means there are people who unnecessarily spend money on car seats (say to replace a broken car seat when a kid gets old enough to ride with a seat belt instead), who could otherwise spend that money on something else that benefits their kids' health (food, medicine, doctor's appointments, etc.), or other good things for their kids (books, exercise). In which case, Economics 101 tells us that laws that force people to spend money on unnecessary car seats also have the inevitable effect of leading to childrens' deaths as well. In a world of scarcity there is never a free lunch. Passing laws that forbid people to take car rides of more than 8 hours or prohibit driving over 30 miles an hour or prohibit people from driving small, light fuel-efficient cars might also reduce the number of casualties to chldren. But there are obvious social tradeoffs on safety, economics, and other concerns for each of these policies. The question is which policy is going to result in fewer overall casualties.

There are plenty of situations where what was originally thought to be true turns out later to be incorrect (Russ Roberts reports that 1/3 of medical conclusions turn out to be incorrect), or where efforts to regulate generate unintended consequences and incentives that defeat the regulation or cause unanticipated side-effects. So we gain little, and potentially lose a lot, when we shut down scientific inquiry on the basis that we are concerned that the public may misuse the information that results.

Let me emphasize--I don't know wither Levitt is correct. But conflating statistical analysis with activist demagoguery and inflammatory rhetoric that his research is going to kill kids isn't going to help us figure out the best policy here.

guest:
wow, I think this is the first time I've agreed with one of your posts
7.26.2005 10:56am
Duncan Frissell (mail):
Classic car seat and risk example from the film Dead Calm in which the former Mrs. Tom Cruise is driving down a dark highway in the rain. Her child is in his car seat in the back of the car. He drops his toy, he unlatches himself and climbs down to the floor. Mom freaks out and turns around and tries to grab the kid. Crash ensues. Kid hurtles through the windscreen. Mom is hospitalized with 'problems'. Dad (Sam Neill) decides they need a cruise to recover. They meet sailboat-based serial killer. Problems continue. An obssesive concern with safety has its risks.
7.26.2005 11:07am
Anonymous jim (www):
Thanks for the pointer to the Freakonomics blog. Does anyone else think it noteworthy that the Today show actually edited out the segment on the Freakonomics position on car seats. Would they edit out a segment on (for example) creationism which is outside the mainstream of scientific thought?
7.26.2005 11:25am
guest:
who knows whether they would edit out creationism, but if they did, it wouldn't be for the same reasons. from the tv show's perspective, if the freakonomics car seat theory is wrong, kids could die; if creationism is wrong, there doesn't seem to be any imminent threat to bodily safety...
7.26.2005 11:43am
Nathan_M (mail):

Thanks for the pointer to the Freakonomics blog. Does anyone else think it noteworthy that the Today show actually edited out the segment on the Freakonomics position on car seats. Would they edit out a segment on (for example) creationism which is outside the mainstream of scientific thought?

I don't think it's fair to say they editted it out. As the Freakanomics blog explains:

When the [London] bombs went off (or failed to go off as it were), the Today Show switched to live coverage across the country. That left Dubner on the cutting room floor in the other time zones.

So no, I don't think it is noteworthy. I expect they would have done the same with any pre-recorded segment, and I expect they cut many other segments that day too.
7.26.2005 12:33pm
Public_Defender:
I agree that legal scholars are more willing to accept debate. As an advocate, I find academics useful to help me understand a legal issue, but I don't want one near me when I step into court.

Academics frequently lack the advocate's willingness and ability to intentionally push the law and the facts in a predetermined direction. Most academics would take this as a compliment.
7.26.2005 12:41pm
Anonymous jim (www):
Nathan,

Thanks, I had missed that. My apologies.
7.26.2005 1:22pm
rbj:
From what I understand of the Freakonomics posts, it isn't even a matter of scarcity; buying a car seat as opposed to a textbook. It seems that the car seats (at least the old ones) were worse protection than seatbelts.
"We evaluated child seat performance based on the child seats implicit to usage in that era and found that car seat belts were more effective for children and infants." I just hope that NTHSA's censoring this information did not cost a child his life.
7.26.2005 1:34pm
M (mail):
Sometimes, of course, presenting emperical results can be "irresponsible and dangerous". An example would be when one has tenative and preliminary results, perhaps reached from only one line of inquary where others will be needed to confirm the point, and where, if one is wrong, the result of following one's suggestions would likely be quite devistating, and one has good reason to think that the tenative and preliminary nature of the results will be ignored when they become widely known as if, for example, they are presented on early morning "news" shows. In such cases one might have a duty to wait until more results are in before publicly presenting one's results. I have no idea of the Levitt/car seat example falls into the case or not. But, this issue is well discussed in Philip Kitcher's book _Science, Truth, and Democracy_ which I strongly recommend to anyone who would like an informed view on such issues.
7.26.2005 1:40pm
Cold Warrior:
The incorrect (or, more accurately, incomplete) evaluation of safety measures is a fascinating topic.

Example: I live in Colorado, a state with no motorcycle helmet law. One of my close relatives is a Highway Patrol officer. A few years ago he was given the choice to give up his patrol car for a motorcycle. He's always been into bikes, so he he took them up on the offer. He never misses the chance to comment on the helmetless idiots passing him on their bikes.

And here's my question: which safety measure provides the greater reduction in risk of serious bodily harm?

A: choosing to ride a motorcylce rather than a large car (usually a Crown Victoria; for the sake of this argument, let's ignore reports of design defects in the Crown Vic).

B: choosing to ride a motorcycle without a helmet vs. choosing to ride a motorcycle with a helmet.

My hunch is that that "safety gain" from "A" is far greater than that from "B." But I don't know if anyone's studied it.

Another quick example: I always read in outdoorsy magazines that anyone doing back country skiiing should always carry a portable beacon to aid rescuers in finding them in case of an avalanche.

Again, which measure provides the greater safety benefit:

A: avoid back country skiiing entirely.
B: still go back country skiing, but start carrying a beacon.

I think the answer to this one is self-evident.

I also had similar notions when my first child was born. It is tantamount to child abuse to leave a 4-month old baby asleep, unattended in its crib, while I make a 15 minute roundtrip (by car) to the local 7-11. But what real risk does the infant run by being left alone sleeping? What real risk is there to the infant's life/health by loading it in a car seat and taking it through traffic on what is (for the infant) an unnecessary trip?
7.26.2005 1:43pm
guest:
Though it goes more to environmental issues, Sunstein's Risk and Reason is a good read on this subject.
7.26.2005 1:57pm
JohnAnnArbor:
Lawyers often overstate "risks" to juries to extract huge jury rewards. Rent an expert ("that safety seat was so unsafe it should be called 'the baby blender'"), make an emotive closing argument (channeling the dead baby's soul, if possible) and you're on your way.

Along the same lines, plaintiff's lawyers routinely strike scientists and engineers from product-liability suits. They don't want anyone trained to think analytically to examine the plaintiff's usually exaggerated claims of harm, or examine whether the harm is actually the fault of the defendent.
7.26.2005 2:39pm
lucia (mail) (www):
You missed the real kicker. The doctors tag this ominous note at the end of the letter criticizing Levitt.
We hope that this misleading article does not cost a child his life.



However, earlier in the letter, the doctors seem to concede Levitt's conclusions that seat belts are safer does apply in fatal accidents. Their words (with my emphasis):

They examined children in fatal crashes (about 1,200 per year) while ignoring the equally informative data on those in nonfatal crashes (450,000 per year).


If both sides agree that the data show seat belts are safer in fatal crashes, then the ominous threat kids might die if they wear seat belts seems a bit, shall we say, mis-placed?
7.26.2005 3:09pm
JohnAnnArbor:
Sentence should have been:
"Along the same lines, plaintiff's lawyers routinely strike scientists and engineers from product-liability suit juries."
7.26.2005 4:01pm
Splunge (mail):
I think that law professors may be more comfortable with intellectual disagreement.

From other law professors, yes. And doctors are pretty comfortable with intellectual disagreement from other doctors, too. But if you want to compare oranges to oranges, you need to ask whether law professors are more comfortable than doctors with intellectual disagreement articulated by amateurs. If, for example, an engineer with zero law training stands up in a seminar on product-liability law and persistently contradicts a law prof on how the relevant law should best be written, how is this received? With the generous intellectual respect you suggest -- and I agree -- a fellow law prof would receive, were the argument to play out on blawgs? Or perhaps, um, with a touch of the irascible condescension you might find when docs are told by economists and law professors how to practise medicine?

All professionals are fairly similar in this fault, in my experience, in that they all tend to turn a jaundiced eye on amateur opinion. I'm not even sure that's unreasonable. We Americans tend to be a pain in the arse to professionals with our arrogant amateurism. We have a pretty strong tendency to figure general life experience and a few hours hard thought are just as good as five years graduate education and fifteen years practical experience.

Hence we often feel comfortable harshly criticizing, from our armchair perspective, the engineers who build Space Shuttles, the generals who fight wars, the Justices who decide constitutional law, the economists who set monetary policy, the docs at the FDA who approve or disapprove new drugs, and on and on, right down to the athletes who throw balls through hoops and such.

Sometimes we're right, of course. Sometimes ordinary folk do have a clearer view of legal issues than Supreme Court justices (cf. Kelo, ha ha). And sometimes economists do have better insight than physicians into how advice on child car safety plays out in the real world.

But it's not the way to bet.
7.26.2005 5:18pm
scarhill:
Splunge wrote:
And sometimes economists do have better insight than physicians into how advice on child car safety plays out in the real world.

But it's not the way to bet.


If your example involved selecting treatments for children injured in car accidents, I might agree. When it comes to analysing data about car seat safety, why would you expect that physicians would be better at it than economists? Economists, like other social scientists, deal with the issues around drawing reasonable conclusions from real-world datasets all the time. Most physicians don't.

With respect to your other point about accepting criticism, my experience is that many physicians tend to be poor at handling intellectual disagreement from non-physicians, regardless of the relative levels of expertise in the subject at hand.

Jim
7.26.2005 6:09pm
Anderson (mail) (www):
I'm a little confused on the practicalities. Car seats are no safer than seat belts for a 2-year-old? But the seats I've seen for 2YO's are basically designed to *let* the kid wear a seat belt, said belt being designed for a much taller person. In other words, there may be a bit of a false dichotomy here.

Anyway, I'm confident that bringing Leavitt's paper home to my wife would have no effect whatsoever, so the net effect on my car-seat practice is nil.
7.26.2005 7:24pm
Splunge (mail):
If your example involved selecting treatments for children injured...When it comes to analysing data about car seat safety...

Mmm, well, what I suggested is that an important issue was not the quality of data analyses per se, but, given the data and studies et cetera now existent, what sort of advice should be provided to parents about car seats, and what kind of public debate or skepticism ought to be encouraged about car seats?

These are practical questions, not academic. Being an expert in SPSS or the meaning of a chi-square test is not going to be nearly as much help as, say, several decades of experience with real parents and their actual (not theoretical) response to safety advice. I'm not suggesting that the physicians are right, mind you -- only that the burden of proof that they are wrong would seem to lie with the economics professor and the journalist.

The practical implications of the debate are not minor issues, either. Let me present one speculation to illustrate this: Suppose it turns out that, if parents read in the NYT that the notion that car seats reduce child injury in a crash is considered debatable by some university professors with Ph.D.s who use big words, then large numbers of parents (however illogically) tend to draw the conclusion that the entire idea of restraint is possibly faulty.

This is not wholly implausible, as the debate over seat belts themselves showed. Forty years ago it was a not uncommon belief that it was better to be unbelted in a collision than belted, so that you would have a better chance of being thrown clear of the immediate nexus of the collision. That is, one of the key premises of the child car seat -- the idea that it's best to be restrained in a collision -- is not common sense, and if all one really grasps is that car seats are doubted, it is not so implausible that it is this premise that will come to be doubted.

If my speculation is true, it might be that encouraging public debate or skepticism over car seats leads to decreased child safety, at least in the short run. One would, of course, need to balance this against child safety in the long run, given the plausible argument by the economist that public debate over car seats encourages better data collection and analysis, and better car seats, and better laws.

Another "real-world" example of this kind of thing is the response of parents to fringe debates about vaccines (they cause autism, et cetera). Combine this with the fact that the rare cases where a child does have a terrible reaction to a vaccine are highly publicized, by journalists with newspapers to sell, and the fact that thousands of children succumbing to polio or whooping cough epidemics is not even a distant memory, and the result can be that parents have a highly distorted view of the risk-benefit trade-offs with respect to vaccines.

In that climate, would it be wise for, say, a Congressman to open hearings into whether vaccines do, as certain oddballs suggest, actually cause autism? There is some small chance they are right, after all. Should we not have a public debate over it? Recalling the Alar scare, the children on milk cartons stranger abduction scare, and so on, I'd have to say no. It's not unlike yelling "I smell smoke!" in a crowded theater before you ascertain whether your neighbor is just surreptitiously smoking a cigarette.

Economists, like other social scientists, deal with the issues around drawing reasonable conclusions from real-world datasets all the time. Most physicians don't.

Say what? Is the constellation of symptoms presented by a patient not a "real-world dataset"? Is a sound clinical diagnosis not an example of a "reasonable conclusion" drawn from incomplete and in places erroneous real-world data? I'm quite at a loss to understand this statement.

...my experience is that many physicians tend to be poor at handling intellectual disagreement from non-physicians, regardless of the relative levels of expertise in the subject at hand.

Well, given that you hold fairly negative views of the intellectual integrity of physicians, vide supra, it would hardly be surprising were exchanges of conflicting views to be frosty.
7.26.2005 7:53pm
scarhill:
Splunge wrote:
Mmm, well, what I suggested is that an important issue was not the quality of data analyses per se, but, given the data and studies et cetera now existent, what sort of advice should be provided to parents about car seats, and what kind of public debate or skepticism ought to be encouraged about car seats?

No, you said that physicians are uncomfortable with "intellectual disagreement articulated by amateurs". Do you consider Professor Leavitt to be an amateur when it comes to analysing that sort of question?

These are practical questions, not academic. Being an expert in SPSS or the meaning of a chi-square test is not going to be nearly as much help as, say, several decades of experience with real parents and their actual (not theoretical) response to safety advice. I'm not suggesting that the physicians are right, mind you -- only that the burden of proof that they are wrong would seem to lie with the economics professor and the journalist.

That might be true if Leavitt were making recommendations about whether parents should use car seats. He wasn't.

The practical implications of the debate are not minor issues, either. Let me present one speculation to illustrate this: Suppose it turns out that, if parents read in the NYT that the notion that car seats reduce child injury in a crash is considered debatable by some university professors with Ph.D.s who use big words, then large numbers of parents (however illogically) tend to draw the conclusion that the entire idea of restraint is possibly faulty.

Yes, I'm sure the demographic that reads the NYT Sunday Magazine is very likely to be cowed by "Ph.D.s who use big words". But assume for the moment that that's true. Is your position that Prof. Leavitt shouldn't talk about his results because someone, somewhere might draw an illogical conclusion from them?

Is the constellation of symptoms presented by a patient not a "real-world dataset"? Is a sound clinical diagnosis not an example of a "reasonable conclusion" drawn from incomplete and in places erroneous real-world data? I'm quite at a loss to understand this statement.

Sorry, I meant dataset in the statistical sense of "a list of research subjects and the data vector associated with each." I don't doubt that physicians are skilled at making diagnoses. I just don't think that skill necessarily makes them more qualified than an economist to conduct the kind of study in question.

Well, given that you hold fairly negative views of the intellectual integrity of physicians, vide supra, it would hardly be surprising were exchanges of conflicting views to be frosty.

I have great regard for the intellectual integrity and abilities of most physicians. I don't see what in my original post would cause you to conclude otherwise.

Speaking of intellectual integrity, read the two quotes in the Alex Tabbarok post linked in the original article and consider which researcher has more of it.

Jim
7.26.2005 9:56pm
David M. Nieporent (www):
Say what? Is the constellation of symptoms presented by a patient not a "real-world dataset"? Is a sound clinical diagnosis not an example of a "reasonable conclusion" drawn from incomplete and in places erroneous real-world data? I'm quite at a loss to understand this statement.

No, it isn't a "real world dataset." It's a real world anecdote.
7.26.2005 10:35pm
Splunge (mail):
Do you consider Professor Leavitt to be an amateur when it comes to analysing that sort of question?

Which question, eh? There's the rub. If the question is: "What statistical conclusions do the data permit?" then no doubt he's good enough.

But if the question is: "How should this debate be described in a public forum in order to minimize the chances of many parents drawing wrong and possibly dangerous conclusions?" then, yeah, I'd say he's an amateur. I mean, unless he has an RN and moonlights at a Hyde Park free clinic three days a week.

I realize you and Professor Levitt would like to narrow the issue to where your case is strong, which is to say whether his analysis of the data is statistically correct. By contrast, the physicians would want to expand the issue, to whether he was being fully responsible to publish his analysis in the way he did. I am not necessarily siding with that point of view, but I am arguing that the point is germane, and that physicians have expertise enough that their opinion should a priori be given great weight, and that, if their opinion is instead treated with contempt by amateurs (amateurs in the area of knowing the influence of the media on patient behaviour), then, yes, it's hardly surprising they would react badly.

Is your position that Prof. Leavitt shouldn't talk about his results because someone, somewhere might draw an illogical conclusion from them?

Heck no. But if millions of parents everywhere are probably going to draw dangerously wrong conclusions, then, yeah, I'd say a decent standard of ethics demands he be quite careful about how and where he talks about them, seek professional advice (e.g. from physicians on the front line) first, and so forth.

See, it's all in the adjectives. Choose yours ("someone, somewhere, might, illogical") and restraint looks fussbudgetly silly. Choose mine ("millions, everywhere, probably, dangerous") and it's a horse of quite another color.

Now, who decides which adjectives are the more apt? You? Me? Professor Levitt? Or, hmm, maybe someone with front-line experience in knowing how printed media stories on child safety issues actually influence the behaviour of actual parents?

The whole issue of social responsibility in publishing may be a less familiar to economics professors but medical researchers have to deal with this all the time. Exempli gratia, you start a clinical trial and have some very exciting news -- do you zap it into print, because it will save lives? Or do you wait, finish the study carefully, knowingly let people die so that you can be very certain there is no chance you might mislead?

Or, again, an otherwise healthy man reaches 55, and as his physician you must choose whether to advise him to get a PSA test. There are some quite difficult ethical issues there, in whether in this case more information is really an unalloyed good. Physicians are trained to consider quite carefully the impact on laymen of their opinions and the way they express them, and it's a highly relevant issue almost daily in their work. Could you say the same thing about economics professors? If not -- well, then, maybe when some physicians say a public opinion on the merit of child safety seats should have been more carefully phrased, maybe that's an opinion worth taking very seriously indeed.

That's my point. If yours is that Professor Levitt should continue his research unhindered by any political correctness, and freely publish his findings in scientific journals, and challenge in professional venues the medical establishment in novel ways to buttress their conclusions with solid fact -- you need argue no further, as I quite agree.
7.27.2005 12:05am
Joel B.:
Of course, if you live in California as a result of all the science we're going to end up being required to be in car seats until you're 23 or 200 lbs. Oh wait, I guess that's just booster seats...

(I am curious what the science on booster seats is, and actually my impression is that it's 6 or 60 lbs, but yikes...it keeps going up!)
7.27.2005 12:11am
Cheburashka (mail):
And sometimes economists do have better insight than physicians into how advice on child car safety plays out in the real world.

But it's not the way to bet.


When the question is one of resource allocation - as all questions of public policy necessarily are - economics will always produce necessary insights.

Letting the doctors decide how to allocate health care dollars is how we got into our present health care mess.
7.27.2005 3:52pm
lucia (mail) (www):
Heck no. But if millions of parents everywhere are probably going to draw dangerously wrong conclusions, then, yeah, I'd say a decent standard of ethics demands he be quite careful about how and where he talks about them, seek professional advice (e.g. from physicians on the front line) first, and so forth.

But no one has shown that even one parent anywhere is remotely likely to draw the incorrect conclusion and decide to stop using seat belts all together as a result of being told they are the safest of three options!

You have simply speculated someone somewhere might reason this way.

Frankly, it seems unlikely that Dr. Levitt's analysis showing seat belts are the safest option would lead to huge
number of parents to decide to avoid them.
7.27.2005 7:08pm