pageok
pageok
pageok
Revisiting the Lancet Study on Iraqi War Deaths:

In 2006, the Lancet published a controversial study finding a substantial, continuing Iraqi death toll in the years following the 2003 U.S. invasion. The study bolstered critics of the Iraq war and prompted substantial debate, online and elsewhere.

Neil Munro revisits the Lancet study in the new issue of National Journal.

In the ensuing year, numerous skeptics have identified various weaknesses with the study's methodology and conclusions. Political blogs and academic journals have registered and responded to the objections in a debate that has been simultaneously arcane and predictable. The arguments are arcane because that is the nature of statistical analysis. They are predictable because that is the nature of today's polarized political discourse, with liberals defending the Lancet study and conservatives contesting it.

How to explain the enormous discrepancy between The Lancet's estimation of Iraqi war deaths and those from studies that used other methodologies? For starters, the authors of the Lancet study followed a model that ensured that even minor components of the data, when extrapolated over the whole population, would yield huge differences in the death toll. Skeptical commentators have highlighted questionable assumptions, implausible data, and ideological leanings among the authors, Gilbert Burnham, Riyadh Lafta, and Les Roberts.

I did not follow the debate closely enough to reach a conclusion about the merits of the study or its critics. The Munro article provides a convenient overviewof the controversy for those of us without the time or patience to wade into the depths of the debate. Munro is not entirely neutral, however, as he concludes there are potential problems with the initial study.
Over the past several months, National Journal has examined the 2006 Lancet article, and another [PDF] that some of the same authors published in 2004; probed the problems of estimating wartime mortality rates; and interviewed the authors and their critics. NJ has identified potential problems with the research that fall under three broad headings: 1) possible flaws in the design and execution of the study; 2) a lack of transparency in the data, which has raised suspicions of fraud; and 3) political preferences held by the authors and the funders, which include George Soros's Open Society Institute.
Of these critiques, I find the political preferences of the authors and their funders to be the least persuasive. Political bias of this sort could certainly explain problems with the study, such as a failure to scrutinize sources and ensure their reliability, but I do not think that the authors' ideological predispositions (or those of the funders) should, in and of themselves, case doubts on the study's findings. The Lancet study's conclusions should stand or fall on their own. In this regard, it is interesting that Munro reports the Lancet editors are less confident of the analysis than they once had been.
Today, the journal's editor tacitly concedes discomfort with the Iraqi death estimates. "Anything [the authors] can do to strengthen the credibility of the Lancet paper," Horton told NJ, "would be very welcome." If clear evidence of misconduct is presented to The Lancet, "we would be happy to go ask the authors and the institution for an official inquiry, and we would then abide by the conclusion of that inquiry."

dearieme:
I was agin the Iraq war from the first. I dismissed the Lancet study from the moment I read about it. I am not a "liberal" in the American sense. What's common to these three positions? The ability to recognise bollocks.
1.5.2008 2:56pm
WHOI Jacket:
I always wondered how anyone could claim with a straight face that, in this day and age and with all our technological and tactical advances, we'd killed more than the Conventional and NUCLEAR attacks on Hiroshima, Nagasaki, Tokyo, Dresden, Hamburg and Okinawa combined?

More than the number killed in the American Civil War on BOTH SIDES?
1.5.2008 3:04pm
statfan (mail):
Please keep in mind that it's not people killed. It's excess deaths. That's things like kids who couldn't get medical treatment because the hospital had been blown up, or the doctor had fled, or the medicine wasn't available.
1.5.2008 3:31pm
Bruce Hayden (mail) (www):
Please keep in mind that it's not people killed. It's excess deaths. That's things like kids who couldn't get medical treatment because the hospital had been blown up, or the doctor had fled, or the medicine wasn't available.
Well, if you are going to define it that way, then you should logically subtract all the people who would be dead if we had not intervened. For example, how about all the people who were treated by American (etc.) doctors or corpsmen? Or all the kids who are now vaccinated for the first time? You should also probably deduct Saddam Hussein's annual death toll by torture, gas, murder, etc.
1.5.2008 4:06pm
Zed:
Bruce,

In fact, that was done. The comparison was of mortality rates before the war and after, IIRC.
1.5.2008 4:26pm
Zed:
I retract my last statement. Upon a quick review, it appears that I'm thinking of a different study.
1.5.2008 4:47pm
frankcross (mail):
The numbers seem awfully high. But in fact the ratio of actual deaths to officially counted deaths is not that different than in other catastrophes, such as famines.

The apparent lack of data availability troubles me. While it's not uncommon for those who have collected so much data to decline to share them promptly, given the public prominence of this article, I think they should have released them. I of course doubt they made up the numbers, but the failure to release somewhat limits the ability of others to screen for methodological problems.
1.5.2008 5:37pm
cirby (mail):

But in fact the ratio of actual deaths to officially counted deaths is not that different than in other catastrophes, such as famines.


The problem with the "ratio of officially counted deaths" is that the Lancet study claimed to have actual death certificates for well over 90% of the deaths reported in the study.

Which would also suggest (if their methodology was anywhere near correct) that the Iraqi government should have records of almost ALL of those 600,000 to 1,000,000 deaths. Which they don't, by any stretch...
1.5.2008 7:17pm
Eli Rabett (www):
The survey authors have continually asked the US to do a survey, the constant response is "we don't do body counts" [T. Franks] which should tell you something.

Good discussions about the statistics can be found at Deltoid, where it is clearly established that David Kane doesn't know anything about statistics. (see especially the comments of Robert, a well known statistician)
1.5.2008 7:20pm
WHOI Jacket:
The Army doesn't do body counts anymore because they were raked over the coals for doing them in Vietnam.

And when figures were postulated a year or so ago, leftists in Salon, ThinkProgress, The Age, The Independent, etc. etc (just google "army enemy body counts"), they raised Cain again.
1.5.2008 7:36pm
Robb Shecter (denk) (mail):

I always wondered how anyone could claim with a straight face that [...] we'd killed more than [...] More than the number killed in [...]


Sounds like the logical fallacy of Appeal to Incredulity.
1.5.2008 7:37pm
WHOI Jacket:
Appeal to Incredulity That's around 500 a day over 4 years! The bodies should be stacked like cord wood.

It shouldn't take the magic of statistics to notice something like this.
1.5.2008 7:43pm
PersonFromPorlock:

Good discussions about the statistics can be found at Deltoid, where it is clearly established that David Kane doesn't know anything about statistics. (see especially the comments of Robert, a well known statistician)

Well, it's a good discussion if you enjoy watching experts, or people claiming to be experts, berate each other. Also, 'well known statisticians' generally have last names.
1.5.2008 7:56pm
Deech56 (mail):
PersonFromPorlock, if you had clicked on the link that Eli so kindly provided, you would have found out that Robert does indeed have a last name: Chung. The threads were no so much a battle between experts, but a serious smackdown.
1.5.2008 8:16pm
Eli Rabett (www):
Hmm, the second link was to a post entitled Robert Chung on David Kane, and, a moment's google gives you a list of his qualifications.

Yes, body counts of civilians would be so embarrassing.
1.5.2008 8:19pm
Jayhawk10:
Robert Chung, by the way, is a demographer who teaches at Sciences PO in Paris (one of the major European centers for research in the social sciences):

http://mpa.sciences-po.fr/sub500/sub50106.html

In other words, I would assume he has some idea of what he's talking about when he discusses how to estimate the death toll of the Iraq war.
1.5.2008 8:20pm
Jayhawk10:
It appears Eli Rabett and I, if not brilliant, at least think alike. But yes, it's undeniable that Chung has substantial academic qualifications in the areas relevant to this topic.
1.5.2008 8:23pm
WHOI Jacket:
I don't care about credentials, I want some sort of ground truth. To take that number is to say that 1 out every 42 people in Iraq (pop. 27,500,000 per World Factbook) is dead. 2.4% of the population, or a little more than than 1% lower than the ratio of Japanese killed in WW2 (3.75%). Compare to Italy (1.04%) or Great Britain under the Blitz and waging Total War (0.94%)
1.5.2008 8:33pm
Oren:
Well, if you are going to define it that way, then you should logically subtract all the people who would be dead if we had not intervened.
You know, for all the awfulness of Saddam's regime (which I don't minimize), he didn't really kill/torture/maim all that many people. It seems that feeding his son-in-law into a wood chipper and executing entire towns every once in a while really obviates the need for any more violence. It may be, in fact, this awfulness (again, not to minimize it) that actually prevented the sort of free-for-all we have to day.
1.5.2008 8:44pm
Jayhawk10:
Saddam Hussein probably ordered the deaths of hundreds of thousands of his own citizens. Contrary to Oren, that doesn't make me think that he "didn't really kill... all that many people." But Oren's basic point has some merit. At the time of the U. S. invasion, Iraq appears to have been quite peaceful. This was at least in part due to Saddam's ruthlessness at suppressing his opposition. The political void that appeared with his overthrow has resulted in a conflict that, at a conservative estimate, has already claimed 100,000 lives, and quite possibly many more.
1.5.2008 8:51pm
Oren:
Jayhhawk, are you talking about the Iran-Iraq war or just the internal Iraq stuff, because I highly doubt the latter hit 100,000. Then again, the former was entirely his doing (although I suppose we get blame as enablers) so we can count that against him, in which case hundreds of thousands is reasonable.
1.5.2008 8:55pm
Anonymous 1234:
It is important to remember that this Lancet study was done with all the honesty and integrity you can expect from "Science". This is the Lancet. As "Scientific" as you get.

Global warming is a "science". Evolution is "science". In other words they are both garbage and the mad rantings of the liberal "enlightenment" mind.

Science is dead, and this "study" is just another nail in its coffin.
1.5.2008 9:01pm
Jayhawk10:
Oren, I wasn't including the Iran-Iraq war. Saddam's war crimes prosecutors accused him of killing between 100,00 and 200,000 Kurds in the 1987-88 massacres. Throw in all of the other victims of his regime over the couple of decades he was in power, and I'd guess you'd be well over 200,000.
1.5.2008 9:04pm
Jayhawk10:
Anonymous, I think your point is very well-taken. The following thread from Deltoid has quite a bit of discussion about statistics, research standards in epidemiology and the inability of Lancet critics like David Kane to understand them:

http://scienceblogs.com/deltoid/2007/08/
robert_chung_on_david_kane.php

As Robert Chung points out in the thread, this doesn't mean the conclusions of the Lancet study are certainly correct. But many of its critics seem to have violated basic standards of good faith in academic argument.
1.5.2008 9:09pm
NicholasV (mail) (www):
Deltoid? Puh-lease. Tim Lambert doesn't have any problems straight out lying about various scientific issues (e.g. climate studies) - as anyone who regularly reads Climate Audit knows - and I'm not talking about debatable stuff, he flat out says things which can be shown to be not true. Why would I trust him on yet another highly politicized subject?
1.5.2008 9:45pm
NicholasV (mail) (www):
Perhaps I should be more clear - he has, on more than one occasion, vehemently defended the Mann, Bradley &Hughes 1998 study, which Steve McIntyre, an expert statistician, has shown to have multiple fatal flaws which completely invalidate it. He has had these flaws pointed out, in detail, multiple times and yet continues to act as if the study is valid. See climateaudit.org for more details if you are so inclined.

If he so vehemently defends a study which has such deeply flawed statistics, should I really care what he has to say about yet another controversial statistical study? OK, I'm sure that's some kind of logical fallacy that someone will point out, but frankly I don't want to waste my time with him and I'm surprised anybody else does.

I can cite other examples where he's let his political leanings get in the way of facts but that one is so devastating I'll leave it there.
1.5.2008 9:49pm
Jayhawk10:
Tim Lambert isn't the main discussant on the thread I linked to, and it's pretty clear from the discussion (if one would take the time to read it) that David Kane just doesn't know half of what he's talking about when he criticizes the Lancet study. Does that mean the study doesn't have flaws? No, but reporting like Munro's unfortunately hasn't done a good job in distinguishing serious criticisms from the flawed claims made by Kane.
1.5.2008 10:05pm
Mike Friedman (mail):

Political bias of this sort could certainly explain problems with the study, such as a failure to scrutinize sources and ensure their reliability, but I do not think that the authors' ideological predispositions (or those of the funders) should, in and of themselves, case doubts on the study's findings.


They certainly have some bearing.

For example, if the authors strongly supported the Iraq war I would tend to be skeptical of claims that they overcounted deaths whereas since they did, in fact, oppose it I would be skeptical of any claim that they undercounted.
1.5.2008 10:41pm
frankcross (mail):
The study looks good. People in this thread who trash it have no actual arguments against its methodology.

But I think they need to release everything. And it concerns me a little that they won't.
1.5.2008 10:57pm
Bart (mail):
Of these critiques, I find the political preferences of the authors and their funders to be the least persuasive. Political bias of this sort could certainly explain problems with the study, such as a failure to scrutinize sources and ensure their reliability, but I do not think that the authors' ideological predispositions (or those of the funders) should, in and of themselves, case doubts on the study's findings.

C'mon now. Without having seen the results, there is ample evidence of bias. The financiers, authors and the magazine editors are all outspoken and often virulent anti-American and anti-war advocates. Both "studies" were timed specifically for release before the 2004 and 2006 US elections. The peer review of the first "study" was sacrificed to ensure that it would be released before the 2004 election.

Then there is this: Mr. Roberts tries to go to Washington. Roberts, who opposed removing Saddam from power, is the most politically outspoken of the authors. He initiated the first Lancet study and repeatedly used its conclusions to criticize Bush. "I consider myself an advocate," Roberts told an interviewer in early 2007. "When you start working documenting events in war, the public health response -- the most important public health response -- is ending the war."

This entire episode is an all too familiar academic disgrace.

As a trial attorney defending against personal injury and construction defect cases, I have discredited dozens of so called scientific or medical experts who prostitute science for ideology or money. I have an extremely cynical view of the science on nearly any politically controversial topic. From alar to global warning, the list of garbage science produced for ideology or money is long and ignominious.

The Lancet studies are one of the most blatant examples of politicized science.
1.5.2008 11:14pm
AnonLawStudent:
Just to throw a bit of reference on the fire:

In response to strong criticism regarding his methodology, e.g. 369 LANCET 101 (2007), Roberts promised to provide the data and descriptions of the methodology to other researchers, see, e.g., 446 NATURE 6 (2007). However, Roberts is only providing the information to "certain" researchers, and has refused to release it to well-qualified critics, 316 SCIENCE 355 (2007). One of the most damning critiques of the Roberts study is that it is hard to critique in the absence of data and methodology. For those of you defending it, on what basis do you have for saying that it "looks good"? In science, the burden is on those seeking to defend a conjecture, not those seeking to refute it.
1.5.2008 11:41pm
MnZ:

For those of you defending it, on what basis do you have for saying that it "looks good"?


The results are their basis...
1.5.2008 11:51pm
Jayhawk10:
AnonLawStudent, as explained in the earlier discussion on Deltoid, the data that the Lancet authors have refused to make available to other researchers would, if made public, likely violate the confidentiality agreements entered into with the participants in the study. The basic data sets and computations were disclosed to critics like David Kane.

For the record, I don't claim the study looks flawless. The number of data clusters, as Munro points out, might be too low. The problematic conditions in Iraq, which the study blames for massive numbers of deaths, unfortunately also make reliable data collection about mortality rates very difficult, and leave the door open to sampling bias or fraud. The official mortality figures, though, are subject to the same sorts of concerns. Furthermore, the basic conclusion of the Lancet study, namely that the real number of deaths in Iraq since the U. S. invasion began are dramatically understated by the official figures, has been corroborated by other studies. Thus, while the jury may still be out on the Lancet estimate, I don't think it's time to convict of fatal bias just yet.

Finally, we shouldn't forget that even a conservative estimate of the number of people to die as a direct result of the U. S. invasion is still around 100,000 people. Awfully large number of people to kill over some bad stovepiped intelligence, wouldn't you say?
1.6.2008 12:20am
NicholasV (mail) (www):
Am I right in remembering that the 95% confidence interval on their results was in the range of 10,000-650,000?

In other words, the 650,000 figure was the upper end of a range spanning six orders of magnitude?

If so, I'd argue that all the study can conclude is that they don't really have any clue what the real figure might be with any reasonable level of confidence.
1.6.2008 12:43am
NicholasV (mail) (www):
OK I guess I am using "order of magnitude" differently than many people. What I mean is, you have to double the lower bound six times before you get near the upper bound. (That would actually be 640,000). I believe this is known in certain statistical circles as a "floor to ceiling confidence interval".

If you use the more common definition of order of magnitude, a ratio of ten, it spans about two orders of magnitude. Still an awful lot, considering that they're only 95% confident that the real value is between those ranges.
1.6.2008 12:49am
Harry Eagar (mail):
'The study looks good'

Sure, like Life magazine's readership claims looked good.

They're claiming that each sampled item was a stand-in for 2K events.

Too fine, even if the sampling was carefully conducted, which it wasn't.
1.6.2008 1:21am
Jayhawk10:
Nicholas, the 2006 Lancet Study estimated that the 95% confidence interval for the number of excess deaths following the U. S. invasion up to June 06 was 392,979-942,636. Violent deaths were estimated to probably comprise around two-thirds of the total number of excess deaths. The weak point of the study, as I've already indicated, is not the statistical methodology used, but the accuracy and representativeness of the initial data. Given the very real concerns about anonymity that participants in the study likely had, I'm not sure how much it'll be possible to investigate that data. The 2007 QBR study gave similar findings to the Lancet study, though, so that gives me some reason to think it might be on the right track.

Given that I was able to hunt down that information in about 30 seconds online, I have to wonder about Nicholas' motivations in criticizing the Lancet study as much as I do Kane's. Of course, Nicholas earlier referred to Steve McIntyre, whom a brief trip to Wikipedia revealed as a long-time mining company employee turned professional global warming skeptic with just a bachelor's in pure math and an Oxford masters in PPE, as an "expert statistician" should probably have already made wonder the same thing.

For the record, I'm a conservative Republican who initially supported the Iraq War. But unlike far too many fellow members of my party, I've never believed in burying my head under the sand when it becomes clear I've made a mistake, whether on foreign policy or the environment.
1.6.2008 1:27am
AnonLawStudent:
Jayhawk10,

The reason that methodology and/or data isn't disclosed is irrelevant; that it isn't disclosed effectively excludes whatever is claimed from the process of science. Consider the following:

INVENTOR: I've done it! A perpetual motion machine! Look at it go! This meter says the energy in equals the energy out.
SCIENTIST: Common sense tells me this is fishy. Let me see how you are measuring energy, and let's look at your raw data.
INVENTOR: Sorry, can't do it. The meter is a trade secret, and I promised the guy who made it that I wouldn't tell anybody how it works or exactly what type of energy it's measuring.

Doesn't sound much like science to me. As for your particular rebuttal of my criticism, forgive me if I defer to four unanswered pages of critiques in one of the world's premier scientific journals, as well as a refusal to provide the relevant data to a London School of Economics professor under a confidentiality agreement, instead of a blog by "a computer scientist at the University of New South Wales." Again, I assert that the burden is on those making the conjecture, not those seeking to refute it.
1.6.2008 1:54am
Jayhawk10:
AnonLawStudent, take a look back at my previous posts: have I anywhere claimed that the nondisclosure of parts of the original data isn't a problem for the Lancet study? As I've said several times, it clearly is. As I stated in one of my earlier posts, the jury is still out on this study.

On the other hand, it's common for personal information to remain unavailable in epidemiological studies. In this case, much of the data requested by the LSE researcher was destroyed soon after collection to ensure anonymity, given the consequences a violation of that could have in Iraq. The criticisms in Science were not unanswered; Burnham and Roberts sent a letter to the editors in response. The discussion in Deltoid was not carried out primarily not by Tim Lambert, but by the demographer Robert Chung. The Lancet paper was the subject for a plenary session at the 2007 meeting of the American Statistical Association, and received cautious approval from statisticians J. Asher and F. Scheur. None of this looks to me like evidence of a paper that's a scientific disgrace.

Of course, the only way to ultimately determine whether a study is reliable is to compare it with carried out by other researchers, e.g. the QRB study. By that measure, the Lancet study may underestimate the number of deaths.
1.6.2008 2:20am
PeterB:
Of these critiques, I find the political preferences of the authors and their funders to be the least persuasive

I disagree.

The Soros-linked group funding was not mentioned in the original study. Soros et al have public policy agendas which they promote.

Undisclosed support by a drug company would cast a real shadow on a drug study. Why shouldn't undisclosed support by a policy group cast doubt on a study that reinforces the group's position?

Economic interest is not the only source of bias.
1.6.2008 4:06am
PeterB:
Of these critiques, I find the political preferences of the authors and their funders to be the least persuasive

I disagree.

The Soros-linked group funding was not mentioned in the original study. Soros et al have public policy agendas which they promote.

Undisclosed support by a drug company would cast a real shadow on a drug study. Why shouldn't undisclosed support by a policy group cast doubt on a study that reinforces the group's position?

Economic interest is not the only source of bias.
1.6.2008 4:06am
Mike G in Corvallis (mail):
Oren wrote:

You know, for all the awfulness of Saddam's regime (which I don't minimize), he didn't really kill/torture/maim all that many people. It seems that feeding his son-in-law into a wood chipper and executing entire towns every once in a while really obviates the need for any more violence. It may be, in fact, this awfulness (again, not to minimize it) that actually prevented the sort of free-for-all we have to day.

statfan wrote:

Please keep in mind that it's not people killed. It's excess deaths. That's things like kids who couldn't get medical treatment because the hospital had been blown up, or the doctor had fled, or the medicine wasn't available.

National Journal article:
Lafta [one of the co-authors of the Lancet studies] had been a child-health official in Saddam Hussein's ministry of health when the ministry was trying to end the international sanctions against Iraq by asserting that many Iraqis were dying from hunger, disease, or cancer caused by spent U.S. depleted-uranium shells remaining from the 1991 Persian Gulf War. In 2000, Lafta authored at least two brief articles contending that U.N. sanctions had caused many deaths by starvation among Iraqi children. In one article, he identified malnutrition as the main contributor to 53 percent of deaths among hospitalized children younger than 2, during a 1997 survey carried out at Saddam Central Teaching Hospital. The article cited no health data from before the sanctions, yet it asserted, "We can conclude from results that the most important and widespread underlying cause of the deterioration of child-health standards in Iraq is the long-term impact of the nonhumanized economic sanction imposed through United Nations resolutions." The article was published in 2000 by the Iraqi Journal of Community Medicine.


If I recall correctly, Lafta's reports were the source of the widely reported (and by some, widely accepted) claim that the pre-war sanctions on Iraq were causing the deaths of "hundreds of thousands" of children.

I really don't see how one can reconcile Oren's assertion of relatively low pre-invasion mortality with the supposed methodology of the Lancet studies or with the supposed studies of pre-war mortality due to the sanctions.

For that matter, putting Lafta in charge of gathering and reporting all of the data for the study strikes me as being rather like putting Baghdad Bob in charge of CNN.
1.6.2008 6:51am
PersonFromPorlock:
Deech56:

PersonFromPorlock, if you had clicked on the link that Eli so kindly provided, you would have found out that Robert does indeed have a last name: Chung.

On the second link, I found:

David Kane asks:

Are you sure that commentator "Robert" is Robert Chung and that he wants his identity revealed here?

To which 'Robert' replies:

I'm pretty comfortable with who I am and I don't think I hide it. Many Deltoid regulars have known it for a while. Besides, exactly who I am is pretty irrelevant--it's funny but irrelevant.

I read this as a cute answer that avoids the question and leaves Robert's identity unknown. I'd also point out that Robert's debating style is remarkably adolescent for an established professional.
1.6.2008 8:17am
NicholasV (mail) (www):
Of course, Nicholas earlier referred to Steve McIntyre, whom a brief trip to Wikipedia revealed as a long-time mining company employee turned professional global warming skeptic with just a bachelor's in pure math and an Oxford masters in PPE, as an "expert statistician" should probably have already made wonder the same thing.


And you believe Wikipedia? Seriously?

For a start, the term "global warming skeptic" is not even close to the mark. He's also not professional - he is retired. If you have read his work, you will know that he is no slouch at statistical analysis either.

I wonder if we read the same study. Was there more than one? I clearly remember the wider confidence interval.
1.6.2008 9:38am
Richard Aubrey (mail):
The Lancet report was 'way too convenient for the sponsors, including Soros, and the guys doing the study, including Lancet.
That, all by itself, should make one skeptical.
I recall that NASA, after being vigorously shaken by one or two amateur experts, admitted to having made some errors in their math about global warming.
And their biggest mouth, who spent about a quarter of his hundreds of interviews claiming he was being silenced, got a huge amount of money from Soros.

Seems to me that if Bush, Rice, Cheney, and whomever else you don't like is supposedly on the Saudi payroll, or following their own path to undeserved riches, then it follows that money must indicate dishonesty. So...? The Lancet study got money, Hansen got money. If it means something in one place, proving it doesn't in another place ought to be required.

Everything is politicized. Extraordinary convenience ought to be a huge red flag.

Did the Lancet article say that the bulk of the casualties were due to US air strikes? If so, then comparing tonnages dropped in Iraq in WW II and Viet Nam is a legitimate question.

Excess deaths includes, as has been said, those due to reduced access to medical care, not simply violent deaths. But those people would be the most fragile, who ought to have been winnowed out by the evil sanctions. What was left of the population would have been like the Spartans.
Something's amiss here. And the disproportion of those dying being men of military age--the least fragile--indicates combat,not old age, heart attacks, and untended athlete's foot. I do not joke about the latter. In nasty, dirty environments, such problems can lead to infections and sepsis.
1.6.2008 10:22am
Eli Rabett (www):
Person from Porlock somehow neglected to mention that the post that Tim Lambert put up is entitled Robert Chung on David Kane, that Chung had self-identified in other comments to other posts as himself, and was so identified in many comments to the post in question and that Kane was simply behaving there as you are here, refusing to accept a clearly established truth.

If you dig through many of the other posts that Lambert has (see first link) on the Lancet surveys, you will find Iraq Body Count represented by a joshd. It's pretty clear who he is also.

BTW if you want a more recent survey, try this one, which puts the death toll from the war over 1.1 million (this is NOT excess deaths such as for the Lancet surveys)
1.6.2008 10:38am
Richard Aubrey (mail):
Eli.
No wonder the gravediggers are so bummed. The war's practically over and the million plus they had as a bonus on top of the ordinary business isn't there any longer.

This is a snarky way of saying there ought to be a million plus graves someplace in addition to the usual. Someplace.


You'll recall the national panic about the epidemic of kidnapped children a decade or so back. Seems there were thousands and thousands and thousands. Some folks, like me, asked where were either the remains, or the teenagers escaping from durance vile with their stories. Should have been stacks of one or the other.
Turns out to be about a hundred stranger abductions a year. The activists had gotten a good portion of society panicked by pointing to custody issues as "kidnapping".
The lesson is that activists will make up whatever numbers they need.
And if you agree with the activists' position, the fact that their latest "STUDY" is a slam-dunk confirmation of your/their position ought to make you very suspicious.
But it won't.
1.6.2008 10:48am
Taltos:
I keep seeing people harping on the whole "sound methodology" nonsense everywhere. Who cares if the methodology is sound when the results are clearly not. I have yet to hear a reasonable explanation for where all of these 500+ a day extra (remember these are supposed to be extra dead on top of whatever the normal mortality rate would be) bodies are ending up outside of the black helicopter roving cremation van varieties.
1.6.2008 11:01am
Michael B (mail):
1) possible flaws in the design and execution of the study
2) a lack of transparency in the data, which has raised suspicions of fraud
3) political preferences held by the authors and the funders, which include George Soros's Open Society Institute
"Of these critiques, I find the political preferences of the authors and their funders to be the least persuasive. Political bias of this sort could certainly explain problems with the study, such as a failure to scrutinize sources and ensure their reliability, but I do not think that the authors' ideological predispositions (or those of the funders) should, in and of themselves, cas[t] doubts on the study's findings."
It shouldn't so much as serve to cast doubts? In this era, in this time? Especially so in the additional light of both of the other two factors?

Prior to the era ideologically and culturally infiltrated and kicked off by the New Left such reasoning might hold, might be sound. But currently and since the soixante-huitards, since '68, such reasoning is perforce abstracted from reality, is dubious in the extreme.
1.6.2008 11:04am
Jayhawk10:
Nicholas, there are two Lancet studies. The first, conducted in 2004, had a wide confidence interval, while the second, conducted in 2006, had the confidence interval I cited previously. You could have found this out for yourself, though, with a few minutes worth of googling. And don't give me the canards that you can't trust Wikipedia or that McIntyre's an independent source on climate change because he's nominally retired from the mining industry. Links on the Wikipedia entry for McIntyre take you to citations on McIntyre's own blog that confirm his connections to mining. McIntyre's consulted for the industry after his official retirement, and life-long employment by mining companies isn't the sort of background that makes me expect objectivity on global warming. That doesn't mean he knows nothing about statistical analysis, but McIntyre doesn't strike me as either an impartial or well-qualified authority on climate change.
1.6.2008 11:12am
Jayhawk10:
Michael, contrary to Adler, I actually view potential bias by the authors of the study, either conscious or unconscious, as the most likely source of errors. The basic statistical methodology appears sound, but the underlying data has proven hard for other investigators to verify. On the other hand, we have the QRB study cited by Eli which appears to estimate an even higher number of deaths, the U. S. military refusing to carry out an investigation of its own, and none of the study's critics wanting to pony up 50,000 or so of their own cash to try to replicate the results. All of that makes me think political bias may have affected the official body counts in Iraq more than it did the Lancet study.
1.6.2008 11:16am
AnonLawStudent:
Eli / Jahawk,

Cite to publication of the "QRB" study in a major journal, versus a media outlet?
1.6.2008 11:34am
Eli Rabett (www):
Well, as a marker, there were 40K UNIDENTIFIED bodies buried in the Valley of the Peace graveyard at Najaf since the US invasion. This graveyard is used by a majority of Shia (estimates are 2/3) in Iraq. Please note that this is a report from an Iraqi government source and that there are probably some Sunni mixed in. otoh UNIDENTIFIED bodies don't have families that pay for the burial so there is a lot of pro bono digging going on.
1.6.2008 11:41am
Eli Rabett (www):
It's ORB which stands for Opinion Research Business. It is a polling company like Zogby. Why do you think that theyt would publish in a journal, does Zogby? Eli takes it from the tenor of your question that you think that the Lancet studies are reliable because they were published in an academic journal and not by for profit. Good that we agree. Or are you just practicing to be a lawyer with a weak case?
1.6.2008 11:50am
Michael B (mail):
Jayhawk,

Just a suggestion, but you might save catastrophic climate change for a different thread - v. here. It's not going to serve you well in terms of reflecting lack of bias or in support of more probative interests still.

As to the other, it invokes a sizeable set of issues, both long term and short term, some difficult to resolve and others nearly, if not actually, impossible to resolve. E.g., I cannot imagine why the military would want to get involved in its own study. If the results would prove to be "good," massive criticisms and rhetorical browbeatings would ensue from every quarter imaginable. If the results would prove to be "bad," what exactly would the contrasting measure be? A comparison with Saddam & Sons' Iraq, both past and in some "alternative universe" future, an unknowable that itself would need to bring such unmeasurable factors as near and longer term stability, both regional and global, into account? What, for example, are the "alternative universe" factors to be assessed if we had not entered the fray in WWI? Or WWII? Or Korea? Or the Truman Doctrine that guided so much of our praxis during the Cold War?

No thank you, I'll take a pass.
1.6.2008 12:13pm
WHOI Jacket:
Regarding bias, one should note that Les Roberts ran for Congress as a anti-war and anti-Bush Democrat (NY 24th District) in the months preceding the publication of the study.
1.6.2008 1:18pm
Elliot123 (mail):
"AnonLawStudent, as explained in the earlier discussion on Deltoid, the data that the Lancet authors have refused to make available to other researchers would, if made public, likely violate the confidentiality agreements entered into with the participants in the study."

If that's the case, and if the data can't be released, then all we have left on which to evaluate the study is our evaluation of the credibility of the authors. That's an intereting standard. On what other issues do we use that standard. Is it sufficient to support accepting the conclusions of their study?
1.6.2008 1:41pm
Bruce Hayden (mail) (www):
After all this hoopla, I am still trying to figure out why, except out of political bias, anyone would accept any study where there is no transparency, or apparently even translucency, of data. Bells went off in my head when I read that the original data was destroyed for confidentiality reasons. Reminds me a bit of that anti-gun study a couple of years ago where none of the data could be validated.

I am sure that if the authors had tried hard enough they could have released their data in such a form that it was sufficiently anonymous to protect whatever confidentiality there might have been. Instead, they appear to have destroyed it. I find this extremely suspicious. They are, in essence, saying "trust us" because we are "scientists". But given the apparent extreme policial biases of at least some of the researchers, why should we trust them? And how can you even check for basic math errors, more-less something substantial, if no one reputable can get at the original data?

No one defending the study here so far has really come up with any reasons why we should accept a study that cannot be verified. Sorry. Until I see more, I will consider it a political hack job pretending to be science.
1.6.2008 1:56pm
Jayhawk10:
Michael, the reason the military might want to carry out a survey on their own is to give the American public and policy-makers accurate information about the costs of the occupation of Iraq. In my book, the military in a democracy has more an obligation to make such information available than it does to try to spin the situation on the ground in favor of the status quo.
1.6.2008 1:59pm
Jayhawk10:
Bruce and Elliot, let's be clear that the Lancet authors have released quite a bit of data. They simply haven't released, and in many cases apparently no longer have, detailed information like street addresses of people interviewed, etc that would allow for independent verification of the data they have produced. As I have already said, this definitely raises the possibility of data tampering at some point in the collection process. But how do scientists try to verify experiments with controversial results? In general, they try to replicate results on their own and not just pour over someone else's data. To understand how reliable the Lancet study is, other investigators have to carry out similar research. We have yet to see this, and as a result the jury is still out on the study's findings.

Quite a few commentators on this thread, however, seem ready to dismiss the Lancet study simply because its authors have a political views and its results do not agree with the casualty estimates compiled by using media and official Iraqi reports. Those sources of information, though, are subject to at least as many sources of bias as the Lancet study. Journalists in Iraq don't have the freedom of movement that would allow them to keep tabs on all the violence in the country, while the Iraqi government is clearly underfunctioning and subject to political bias of its own as well. Thus, the short answer to the question of how many Iraqis have been killed since the U. S. invasion is that no one knows. But until we see more solid scientific studies on the question, that answer won't change.
1.6.2008 2:22pm
Michael B (mail):
Jayhawk,

Both subtly and more directly you are substituting your own spin and platitudes ("the military in a democracy ...") for something more substantial. I might respond with "citizens of the demos should spin less" (albeit with more nuance and sophistication), but you're likely to note that's unfair, or some such response. That's but one of the reasons why I said, no thank you, I'll take a pass. I.e. there are fruitful discussions and then there are less fruitful discussions.

Imo and as Yogi Berra might put it: this is not one of 'em.
1.6.2008 2:43pm
Harry Eagar (mail):
Another reason to be skeptical of the Lancet study is that the range it gives covers a factor of 2. Although the authors say they prefer the middle of the range, there is no statistical reason to do so.

The upper range is nearly a million.

WHOI Jacket made the point with historical comparisons, but here's another.

The upper bound of Lancet is about the same as the usually accepted death toll in the Spanish Civil War. Spain in the late '30s had about the same population as Iraq does today, but since the Kurdish region has had much less death, the comparison means we would have to accept that violence in the violent part of Iraq has been considerably more deadly than it was in Spain.

Seems unlikely.
1.6.2008 3:56pm
AnonLawStudent:
Eli,

While publication in a major journal is a significant step in ensuring reliability of a scientific conclusion, it is not per se indicative of reliability itself. As far as screening, publication in a journal such as The Lancet is indicative of only two things: (i) the study has at least some basis in science, and (ii) covers an interesting topic. In fact, publication is only worthwhile to the degree it enables other knowledgable researchers to critique the work. THAT is the basis of my criticism. The absence of details regarding data and methodology prevents determination of reliability; that is the issue in The Lancet study. The link to the ORB poll is no different. As for the quality of a legal argument, a published "study" that can't be verified would stand a good chance of being tossed via the general prohibition on ipse dixit. See Kumho Tire Co. v. Carmichael. Even in court, the focus is on review of methodology, not publication.
1.6.2008 5:45pm
Eli Rabett (www):
Links to the data tables for the ORB survey can be found at the link I provided. An error was discovered and corrected for by examining these tables (religion of the respondents in Baghdad, see also Deltoid for a discussion). The methodology is standard and was described.

As Jayhawk has pointed out verification in these matters is by repetition, something no one who claims that Lancet I and II and ORB are wrong has wanted to do.

The thing I have been looking for is a report on the total number of burials at Najaf, which should be a pretty good metric.
1.6.2008 7:37pm
Smokey:
The fact that the authors absolutely refuse to allow the public unrestricted access to their data and methodology that the same public has paid for is the reason that Jayhawk10 launched his ad hominem attack against innocent bystanders like Steve McIntyre and Climate Audit.org.

The central issue that routinely concerns Climate Audit is the undeniable fact that the proponents of human-caused global warming [AGW] have consistently refused to allow their taxpayer-paid data, or their methodology, to be disclosed to the folks who pay the freight. This is, of course, dishonest. At the same time, James Hansen, Al Gore, Michael Mann, and the Real Climate propaganda site have all received major funding from George Soros, through MovOn.org.

But none of them - not one of these scientists - will provide unrestricted access to their data and methodology. What reason -- other than fear of a negative peer-review, or of the discovery of outright dishonesty -- would any scientist deliberately hide data that was collected and paid for with taxpayer dollars?

The accusations aimed at McIntyre [of being a *gasp!* retired mining company employee] are simply a psychological projection by those who refuse to allow their own side's data and methodology to be examined for basic truthfulness; we are expected to unquestioningly accept their purported science. For example:
Steve McIntyre, whom a brief trip to Wikipedia revealed as a long-time mining company employee turned professional global warming skeptic with just a bachelor's in pure math and an Oxford masters in PPE...
Someone with "just" a degree in pure Mathematics and an advanced degree in Statistics isn't your ordinary blogger. McIntyre has consistently [and very politely] requested the corroborating data and their statistical methodology from Real Climate, Mann, Hansen and others -- all of whom refuse to cooperate. What does that tell you about the ethics/honesty of those pushing their falsified AGW conjecture?

There is a lot of similarity between the pushers of the "one million+ Iraqis dead because of America" and the "give us lots more money or the climate is gonna kill us all" propagandists.

There's nothing new under the sun. This guy understood what's going on in in both of the instances cited above:

The whole aim of practical politics is to keep the populace alarmed (and hence clamorous to be led to safety) by menacing it with an endless series of hobgoblins, all of them imaginary.

~~ H. L. Mencken
1.6.2008 8:04pm
Elliot123 (mail):
"But how do scientists try to verify experiments with controversial results? In general, they try to replicate results on their own and not just pour over someone else's data."

They usually do both. First they examine and try to understand what the other guys did. They look for errors, recalculate, turn it upside down and inside out. They determine if it's worth persuing. Then they move on to replicating with brand new experiments and data.

"Thus, the short answer to the question of how many Iraqis have been killed since the U. S. invasion is that no one knows. But until we see more solid scientific studies on the question, that answer won't change."

I agree. And the Lancet study doesn't help.
1.6.2008 9:43pm
Harry Eagar (mail):
Not to go off track, but McIntyre has scored 2 coups that only a highly competent statistician could have scored:

Y2K and the NASA temp debacle.

Both were right out there for ANY competent statistician to find, but all the other competent statisticians didn't find 'em and McIntyre did.

Good enough for me.

People who diss McIntyre's competence are only revealing their own.
1.6.2008 9:49pm
neurodoc:
Mike G in Corvallis: For that matter, putting Lafta in charge of gathering and reporting all of the data for the study strikes me as being rather like putting Baghdad Bob in charge of CNN.
Given what Eason Jordan, the prez of CNN, later admitted to in the NYT about CNN's acquiescence to Saddam's censors so the "news" broadcaster could maintain a station in Iraq, I can't figure out where the irony in "Baghdad Bob in charge of CNN" is supposed to begin and end.
1.6.2008 11:47pm
David Matthews (mail):
Back in October of '06, I posted my own critique of the study. I still have seen nothing to contradict my conclusions, and the latest published criticism just adds more weight to the concerns I voiced in point #2 of my summary.

Here's the link.
1.7.2008 12:17am
A.C.:
If the study did have a conscious, political bias, what was the goal? If the majority of the excess deaths (however many) were caused directly by the US, Britain, etc., then they were caused early in the invasion and were long over by the time the study was released. If the majority of the deaths were caused by other factors, from disease to the various "internal wars" that went on, you're still left with the question of what you want the West to DO about all that. The study by itself doesn't give a plan for reducing violence, and could have been used as an argument for a troop surge as well as to support withdrawal.

To me this suggests that any political bias was either unconscious or of the "acting out" variety -- not so much aimed at any practical result, but rather just an expression of generalized moral outrage. There's a lot of that going around.

As for why the military doesn't count bodies, I thought the reason was to avoid making killing into the goal. You tend to get more of whatever thing you count, so keeping a body count might lead to operations that kill the most people even when doing so doesn't make sense.
1.7.2008 2:02am
michael (mail) (www):
My local paper reported another Lancet coup recently: that antipsychotics were no better than placebo in reducing agitation in demented elderly. Regardless of whether or not I agree with the idea of using the antipsychotics as a standard practice, it's clear that except in an oddly selected population this is as wrong a conclusion as 'dogs don't bark more than cats.' The editors of Lancet seem abnormally ready to use their prestige and perhaps usual good sense to push pc agenda.
1.7.2008 2:13am
Dogwood (www):
JayHawk said: To understand how reliable the Lancet study is, other investigators have to carry out similar research. We have yet to see this, and as a result the jury is still out on the study's findings.

This is partially true, but as the National Journal article states, the UN did its own more comprehensive survey in 2004, which produced dramatically different results than the 2004 Lancet I report.

Here is the relevant passage:

By contrast, in a 2004 survey, the United Nations Development Program used many more questioners to visit 2,200 clusters of 10 houses each. This gave the U.N. investigators greater geographical variety and 10 times as many interviews, and produced a figure of about 24,000 excess deaths -- one-quarter the number in the first Lancet study. The Lancet II sample is so small that each violent death recorded translated to 2,000 dead Iraqis overall. The question arises whether the chosen clusters were enough to be truly representative of the entire Iraqi population and therefore a valid data set for extrapolating to nationwide totals.

Given the dramatic difference in results between Lancet I and the UN report, the Lancet II results should be setting off red flags all over the place.

Also, when "scientists" decide to become "activists", then everyone should be suspicious of their results.
1.7.2008 2:44am
Richard Aubrey (mail):
A.C. You ask a logical question about what the purpose of the study was, even admitting bias.

The answer is not supposed, IMO, to be logical. It is to be an emotional, hysterical ratcheting up of BDS, of an illogical, damn-the-consequences insistence on getting out. This would support the goals of the putatively biased studier people. Dump on Bush and get the US out of the ME.

As the NYT admitted some time back, even if, as they expected, our immediate withdrawal resulted in genocide, we should get out. You can't support that kind of view with logical questions of the sort you pose. It has to be an inchoate, emotional repulsion on the part of a sufficient number--not that high--of the electorate.
1.7.2008 6:22am
M. Gross (mail):
What? No mention that the 2006 Lancet Report can't match their 2004 results using a similar methodology?

That's the first thing that I noticed.

Their methodology had been used before. The fact that it finally had something to compare against in Iraq and ended up wrong reveals the methodology itself is probably useless... a fact that probably invalidates a number of previous studies in the same vein.
1.7.2008 3:43pm
Eli Rabett (www):
M., in what way was the 2006 survey inconsistent with the 2004 one. Remember years had passed between the two.
1.7.2008 8:11pm
Jayhawk10:

...the UN did its own more comprehensive survey in 2004, which produced dramatically different results than the 2004 Lancet I report.


Dogwood, this statement is not true, as a quick googling of the relevant information would indicate. The UN study does provide some indication that the actual number of violent deaths in Iraq since the invasion began is toward the low end of the Lancet's estimates, but it's an overstatement to say the results are dramatically different.

The UN estimated the number of violent deaths in Iraq in the first 13 months after the invasion as somewhere between 18,000 and 29,000. The 2006 Lancet study estimated the excess deaths during the first eighteen months of the invasion as between 69,000 and 155,000, with at least half of those deaths ocurring during the last six months of that period. Thus, for example, estimating around 30,000 dead during the first year of the invasion and around 40,000 dead during the first half of the second year is consistent with both studies. Furthermore, this demographic pattern would be consistent with media reports about the upsurge of violence in Iraq at the beginning of 04. Once again, though, it's hard to tell how accurate the Lancet's mortality estimates are after April 04 because we've had almost no comparable studies.
1.8.2008 12:13pm
Jayhawk10:
It doesn't really matter, but just to be clear there's even less discrepancy between the 2004 Lancet study and the UN ILCS, as the former estimated between 8,000 and 192,000 dead, with once again a much higher death rate in 04 than 03.
1.8.2008 1:07pm
Richard Aubrey (mail):
"We're pretty sure that such-and-such is between 100,000 and 1,000,000."
What on earth is anybody supposed to do with that?

There is a non-trivial (love that term) possibility that the excess deaths were outside the published max and min. And no certainty as to where, within the max and min the reality is.
It's useless.
1.8.2008 8:43pm