pageok
pageok
pageok
[Amy Zegart (guest-blogging), September 12, 2007 at 12:50am] Trackbacks
9/11: Hindsight Bias?

Several comments about my 9/11 book SPYING BLIND raise a crucial question: can I really talk about the causes of 9/11 without commmitting hindsight bias? The answer is you bet. 340 reasons explain why.

I was deeply worried about this question, so spent two years tracking what happened to every unclassified intelligence reform recommendation from 1991 to 2001. I found that a dozen unclassified studies examined the CIA, FBI, intelligence overall, and US counterterrorism capabilities during the decade. These weren't obscure little groups, but high profile blue ribbon commissions, government studies (Clinton's reinventing government initiative, the FBI's own strategic plan, to name just 2), and nonpartisan think tank task forces sponsored by places such as the Council on Foreign Relations. Together, these studies issued 340 recommendations for reforming intelligence agencies. But almost none of the suggested fixes were implemented before 9/11. Most -- 268 to be exact, or 79% of the total-- produced no action at all. Zip. Nada. Nothing. Only 35 were fully implemented, and these were mostly "study the problem more" suggestions. Here's the kicker: in retrospect, these pre-9/11 reform recommendations were right on target: 84% focused on just 4 key problems:

1. Information sharing 2. The inability of the CIA, FBI, and other intel agencies to work as a unified team 3. Weaknesses in setting priorities 4. Poor human intelligence.

Sound familiar? These are the same deficiencies the 9/11 Commission and Congressional intellgence committees' Joint Inquiry into 9/11 identified as crucial weaknesses that left us vulnerable.

There's more. Much more. Hindsight bias is all about the historical record. And in this case, the historical record is clear: the CIA Director told Congress publicly that terrorism ranked among the top threats to U.S. national security every year from 1994 (when the threat assessment was first made public) to 2001; President Clinton mentioned terrorism in every State of the Union address from 1994 on; in 1999, Secretary of Defense Cohen even wrote an op-ed to an obscure little paper called the Washington Post in which he explicitly predicted a terrorist attack on American soil. That's just a preview of the highlight reel.

It's a smart and fair question to ask whether we really could have seen it coming before disaster struck. But evidence suggests that years before 9/11, intelligence officials and elected lawmakers were aware of the al Qaeda threat, and understood the imperative for intelligence reform. But they were unable to get the fixes they believed were vitally necessary. Sometimes hindsight isn't as biased as it appears.

Patrick McKenzie (mail):
With all due respect, by that standard workers of the federal government have successfully predicted 4,836 of the last one attacks on American soil. Richard Clarke was very, very vocal about the looming terrorist threat with potential to kill thousands and cause billions in losses -- CYBERterrorism.

I'm also going to take issue with the accuracy of the implication about Clinon's mentioning terrorism in every State of the Union address. The implication is clearly that the government identified and understood the problem. The texts, read with the benefit of hindsight, do not provide evidence for this implication:

1997 -- Clinton devotes one sentence to terrorism, in the context of the Chemical Weapons treaty.

Most of the other mentions concern terrorism as one of a trifecta of law enforcement problems, the other two being organized crime and drug trafficking. Now, granted, as a tactical matter Al Qaeda has been known to benefit from all three of these, but we have the benefit of knowing now that addressing terrorism primarily as a criminal matter costs lives because it forces us to be perpetually reactive rather than proactive in addressing it. Example:

>>
I predict to you, when most of us are long gone, but some time in the next 10 to 20 years, the major security threat this country will face will come from the enemies of the nation state: the narco-traffickers and the terrorists and the organized criminals, who will be organized together, working together, with increasing access to ever-more sophisticated chemical and biological weapons.
>>

Give Clinton credit, he wasn't totally wrong, but he's not nearly as right as your post would have it. The 9/11 attacks didn't rely in the slightest on either organized crime or narco-trafficking. They weren't smuggled in by the Columbia drug cartels, they came in through the front door on student visas, and they weren't financed through organized crime except insofar as that definition includes a titularly soverign government we had to declare outlaw later. They weren't perpetuated by an enemy of the nation-state, they were perpetuated by a group which was the de-facto protectorate of a nation state. The contemporaneous biological attack has been almost totally forgotten in favor of the destructive power of common household cutting implements.

And the recommendations issued? They were recommendations to improve security, but they were aimed at threats that never materialized*: conceptualization of a stateless criminal gang armed with computers and anthrax rather than a transnational (but intimately familiar with governments) ideological movement armed with explosives and boxcutters. This is despite the same movement having gone with Plan A, Blow Up A Building And See What Happens, several times over the interval. That would get mentioned in the SotU address for the year, and then the topic would immediately seque back to crime, WMD, and cyberterrorism.

* Technically, I suppose, they have yet to materialize. They still could. Of course, that is exactly what informs the logic of CYA government reports: if you exhaustively document every possible threat and contingency you will have a piece of paper saying that you did, truly, identify the catastrophe that did happen. The fact that it was buried in a mountain of other equally pressing worries because you were not able to excercise your discretion in identifying which avenues were most likely, or failed in your analysis, won't cost you your career. This is the bureacratic game theory rationale for why the US consistently fails to "connect the dots" -- there are too many dots produced and, seperately, the dots which are thought to be important (and which thus catch further allocation of intelligence and countermeasures resources) are not actually important compared with the dots which are identified and (relatively speaking) ignored.
9.12.2007 3:05am
CDU (mail):

And in this case, the historical record is clear: the CIA Director told Congress publicly that terrorism ranked among the top threats to U.S. national security every year from 1994 (when the threat assessment was first made public) to 2001

The problem with this is that the connotations of "terrorism" is just too broad a category. In particular, many of the connotation and unspoken assumptions that go along with a concern about terrorism are quite different now then they were pre-9/11. When these assessments where made, what sort of terrorism were they concerned about? The kind of measures and precautions that would protect against the next Timothy McVeigh don't necessarily have a lot of overlap with the measures that would protect against Mohamed Atta.

Saying that people were concerned about terrorism from 1994 to 2001 is hindsight bias if you ascribe the
9.12.2007 3:17am
CDU (mail):
Sorry about that, I hit the post button accidentally. Continuing:

Saying that people were concerned about terrorism from 1994 to 2001 is hindsight bias if you assume that the kind of terrorism that they were concerned about then is the same kind that we are concerned about today.
9.12.2007 3:19am
Patrick McKenzie (mail):
It took some doing, but I dug up w/o a reference a copy of Cohen's WaPo editorial. I would not characterize it as you have characterized it.

>>
in 1999, Secretary of Defense Cohen even wrote an op-ed to an obscure little paper called the Washington Post in which he explicitly predicted a terrorist attack on American soil.
>>

That makse it sound like Cohen was right in his prediction. Care to follow that with the actual text of the prediction?

>>
In the past year, dozens of threats to use chemical
or biological weapons in the United States have turned out to be
hoaxes. Someday, one will be real. ...

We are preparing for the possibility of a chemical or biological
attack on American soil because we must. There is not a moment to
lose.

>>

The prediction which Cohen actually wrote, as opposed to your gloss of it, does not appear to have been very prophetic in predicting the September 11th attacks. US security policy was far more concerned with preventing that scenario than with the scenario which actually unfolded. Cohen does identify the first WTC bombing in the editorial, and mentions that they're candidates for... planning a biochem attack. Not blowing up the WTC again, that was emphatically not on the radar.
9.12.2007 3:21am
Curt Johnson (www):
Speaking as a career intelligence analyst (now retired), 9/11 was much more a leadership failure than an intelligence failure. Despite whatever Clinton said, or the DCI told congress, or the SecDef wrote, terrorism — as distinct so-called cyber-terrorism — was not taken seriously by the Clinton administration. John Deutch [Clinton's first appointee as DCI] was actively hostile to it, just as he was active hostile to anything that wasn't his hobby horse du jour. Colleagues who had the misfortune of briefing Deutch on things he was not interesting in had to endure profanity laced tirades about their wasting his time and, if reports are to be believed, the occasional barrage of office supplies. Tenant was an improvement in that at least he didn't cuss and throw things, but he was no phoenix and he had no interest or I think aptitude in pushing agendas that the Administration had little or no interest in. And the fact is that support for analysis of the terrorist threat was not high on Clinton's agenda and so received only lip service.

To speak from direct personal experince, the team of which I was lead analyst — and which specialized in threat assessment and project — proposed time and again to include the terrorist threat in our projects. Our proposals were always initially accepted, but by the kick-off meeting they would be watered down. By the first program review (if not sooner) terrorism was invariably stripped from the scope of the analysis. In the face of our repeated complaints that doing so left our analysis dangerously incomplete, we were invariably told we were right but that the "higher ups", acting on direction from the Administration, did not want those topics looked at. The usual reason advanced was that they wanted to avoid turf battles and that someone else was looking at the problem. As we now know, that was not the case. Lack of interest in, not to say hostility towards — though I suspect there was some of that too, in Berger's case — the terrorist threat prevented it being seriously addressed, over the protests of many at the working level in the Intel Community. (I will briefly note that the schism between the analysts at CIA and their upper management in the late 90s was worse than anything I've ever seen or heard of in my professional career. The atmosphere was one of rank despair and at times out-right mutiny.)

In short, Clinton's domestic response to terrorism was in line with his international response: lob a few cruise missiles at valueless targets to be seen as "doing something" and hope the problem went away. That combined with the deleterious actions he did take — heightening the now infamous "Wall" and a directive that further crippled our already non-too-stellar HUMINT capabilities, among others — effectively guaranteed that the Intel Community would be unable detect the 9/11 attacks or even if they did, that the warning would be dismissed and no action would taken.

So yes, we were "Spying Blind" but only because Bill Clinton tied our hands and put blindfold over our eyes.
9.12.2007 5:45am
fishbane (mail):
Patrick: I know the topic is 9/11, but, still:
In the past year, dozens of threats to use chemical
or biological weapons in the United States have turned out to be
hoaxes. Someday, one will be real


Has everyone forgotten the anthrax attacks that immediately followed? Sometimes it seems that way... I still have yet to see a reasonable explanation of where they came from, why interest was seemingly lost so quickly, or even what was going on.
9.12.2007 6:45am
Lonetown (mail):
Here's the kicker: in retrospect, these pre-9/11 reform recommendations were right on target: 84% focused on just 4 key problems:

1. Information sharing 2. The inability of the CIA, FBI, and other intel agencies to work as a unified team


How does this explain Jamie Gorelicks wall? It seems the Clinton team did all they could to frustrate cooperation between agencies.

Truth is stranger tahn fiction.
9.12.2007 8:46am
David M. Nieporent (www):
With all due respect, by that standard workers of the federal government have successfully predicted 4,836 of the last one attacks on American soil.
Patrick has it right here. To show that it's not hindsight, one has to show more than that someone predicted terrorist attacks; one has to show that someone predicted only terrorist attacks. If one predicts everything, one will eventually be "right," but only in hindsight. The same applies to solutions to the problem -- if one proposes that we do everything imaginable, then one will probably be "right" about how the problem could have been prevented -- but again, only in hindsight.

And another problem is that "blue ribbon commissions" that focus on a particular problem can propose solutions to only that problem; actual policymakers have to make tradeoffs. More HUMINT -- well, who could disagree with that? Nobody, ceteris paribus. But in the real world, where tradeoffs must be made, more HUMINT means less SIGINT. A lot of the recommendations amount to "do a better job," which isn't really a recommendation at all.
9.12.2007 8:50am
mrshl (www):
You people are laudably obtuse. For it not to be hindsight bias she's got to find evidence that we were expecting a particularly grave kind of attack that wasn't chemical or biological?

I think she's suggesting that policy makers, if nothing else, were long aware of the shortcomings she cites as the main culprits in failing to stop the 9/11 attacks. Yes, there's plenty of blame to go around (Bush and Clinton administrations, as well as the intelligence communities). But she's making a very persuasive point here:

We knew we weren't ready. Whether we knew what was coming or not, we damn sure knew we weren't ready for it.
9.12.2007 10:01am
x (mail):
Bullshit... we should have known exactly who and what was coming, starting in 1993.

1993
Feb. 26, New York City: bomb exploded in basement garage of World Trade Center, killing 6 and injuring at least 1,040 others. In 1995, militant Islamist Sheik Omar Abdel Rahman and 9 others were convicted of conspiracy charges, and in 1998, Ramzi Yousef, believed to have been the mastermind, was convicted of the bombing. Al-Qaeda involvement is suspected.

1995
Nov. 13, Riyadh, Saudi Arabia: car bomb exploded at U.S. military headquarters, killing 5 U.S. military servicemen.

1996
June 25, Dhahran, Saudi Arabia: truck bomb exploded outside Khobar Towers military complex, killing 19 American servicemen and injuring hundreds of others. 13 Saudis and a Lebanese, all alleged members of Islamic militant group Hezbollah, were indicted on charges relating to the attack in June 2001.

1998
Aug. 7, Nairobi, Kenya, and Dar es Salaam, Tanzania: truck bombs exploded almost simultaneously near 2 U.S. embassies, killing 224 (213 in Kenya and 11 in Tanzania) and injuring about 4,500. 4 men connected with al-Qaeda 2 of whom had received training at al-Qaeda camps inside Afghanistan, were convicted of the killings in May 2001 and later sentenced to life in prison. A federal grand jury had indicted 22 men in connection with the attacks, including Saudi dissident Osama bin Laden, who remained at large.

2000
Oct. 12, Aden, Yemen: U.S. Navy destroyer USS Cole heavily damaged when a small boat loaded with explosives blew up alongside it. 17 sailors killed. Linked to Osama bin Laden, or members of al-Qaeda terrorist network.
9.12.2007 10:12am
Simon Dodd (mail) (www):
Amy, perhaps I need to drink more coffee before trying to asses a post, but if I read you right, you're arguing that there were 268 policy recommendations made during the Clinton administration that you know from information available to you now (by virtue of the passage of time and the unfurling of events), information that wasn't available to decisionmakers then (for the same reason), ought to have been adopted. How is that not hindsight bias? Your assessment of the utility of those measures (and hence the wisdom of not adopting them) rests wholly on ex post information.
9.12.2007 10:18am
Ken Arromdee:
For it not to be hindsight bias she's got to find evidence that we were expecting a particularly grave kind of attack that wasn't chemical or biological?

Yes, that's the point.

I think she's suggesting that policy makers, if nothing else, were long aware of the shortcomings she cites as the main culprits in failing to stop the 9/11 attacks.

She's "proving" this by cherry-picking statements from a government which has, somewhere, described everything in existence as a shortcoming.
9.12.2007 10:33am
bittern (mail):
Y'all got CYA bias. I remember repeatedly reading analyses warning about U.S. vulnerability. The public didn't believe it or didn't care. I noticed that at the time. Not that I did anything either. When the public doesn't care, they're not going to get the gov't to do much to protect them.
9.12.2007 10:34am
PatHMV (mail) (www):
As I read what Amy wrote, what I hear her saying is that it's not hindsight because all of our civilian leaders said publicly that terrorism was a great threat, but then failed to follow through with the measures which would be necessary to fully analyze and combat the terrorist threat, and she knows this by virtue of extensive study of contemporaneous documents which warned that our intelligence structures were not organized to aggressively fight terrorism.

Am I missing something? That doesn't sound like hindsight to me. It sounds like she looked at the review commissions which were conducted at the time, which said to the political leadership "Hey, guys, you said terrorism is our biggest threat, and we agree, but our agencies are not organized to do what it takes to gather information about terrorist threats, analyze them, and take action." Which studies were all ignored until after 9/11.
9.12.2007 10:53am
PatHMV (mail) (www):
I would observe that I think Amy's results reflect what a waste of time and money "white papers" are in government, because it is rare to find a boss with the intestinal fortitude to implement them.
9.12.2007 11:16am
PatHMV (mail) (www):
Patrick McKenzie... I did not get the sense from reading Amy's post that she was particularly defending President Clinton (or Secretary Cohen for that matter) for "getting it right" and accurately predicting the specific attack which was coming. She was simply pointing out that they publicly identified terrorism as a major threat facing us, then declined to heed a number of contemporaneous warnings that our agencies which would be expected to combat terrorism were not well-organized for that task.
9.12.2007 11:22am
Anderson (mail):
Indeed, a normal person would read Zegart as saying that Clinton (like Bush) "got it wrong," by not providing the presidential push that would have gotten these reforms implemented. Nothing like that can happen without the president's making it a serious priority.
9.12.2007 11:39am
Jows (mail):
American society didn't take the threat seriously. Even after the most devastating attack in our country's history there are a lot of people who don't consider the threat serious enough to take any action. On 9-11-01 there were people claiming that you could only fight terrorism with hugs, or something stupid like that.
9.12.2007 11:42am
PatHMV (mail) (www):
Jows, your point is certainly correct, which is why I've never faulted President Clinton for not taking certain more aggressive actions, like declaring war on the Taliban in Afghanistan. Some actions you can only take when the public is 100% supportive, and the public only sees threats like that after the attack, not before.

But there are plenty of reforms in the intelligence-gathering process which could have been taken without that kind of provocation, reforms that can be implemented if the President and his top staff decide it is sufficiently important to do so. It shouldn't take an attack on American soil to lower some of the bureaucratic barriers and ease some of the turf wars which have arisen over time.
9.12.2007 12:10pm
Lugo:
Bullshit... we should have known exactly who and what was coming, starting in 1993.

Using the examples you provide, what we should have expected was an al qaeda truck bomb attack.

evidence suggests that years before 9/11, intelligence officials and elected lawmakers were aware of the al Qaeda threat, and understood the imperative for intelligence reform. But they were unable to get the fixes they believed were vitally necessary.

There is no "fix" that will ensure that you have what you need to stop something like 9/11, which is access to a person inside al Qaeda. You can even throw a ton of money at the problem, and still there is no guarantee you'll have that humint penetration.
9.12.2007 12:30pm
Lugo:
Some actions you can only take when the public is 100% supportive, and the public only sees threats like that after the attack, not before.

A leader is supposed to lead, not follow public opinion. The President is not a totally passive victim of public opinion, he has plenty of power to shape it. We had been attacked plenty of times by al Qaeda before Clinton left office, and if he had said "we're going to go into Afghanistan and kill those guys" - and explained why - then the public would have gotten behind it.
9.12.2007 12:34pm
Anderson (mail):
But there are plenty of reforms in the intelligence-gathering process which could have been taken without that kind of provocation, reforms that can be implemented if the President and his top staff decide it is sufficiently important to do so. It shouldn't take an attack on American soil to lower some of the bureaucratic barriers and ease some of the turf wars which have arisen over time.

Bingo. I would've liked to see Clinton be more aggressive; OTOH, the one time they *were* aggressive, we got the Sudan missile attacks. And the curs baying for Clinton's impeachment/resignation (Repubs &Dems alike), who had no clue what was going on behind the scenes in counterterrorism, had a barking fit.
9.12.2007 12:44pm
Dan Simon (mail) (www):
What's missing in this discussion is the context in which all those warnings about terrorism and poor intelligence were being generated. The Cold War had recently ended, and with it the raison d'etre for a huge chunk of the intelligence community (not to mention the rest of the defense and national security establishment). Millions of people with careers to protect were scrambling to find some justification for continuing to do work that suddenly seemed a lot less important to the country than it used to be. One of the reasons for all those commissions making all those recommendations is that many friends and allies of the intelligence community were trying to maintain the importance and profile of the intelligence community, by couching the urgent need for "reform" in dire warnings about the dangers of failing to grant the community the money, prestige and priority it had had in the past.

Some of those warnings had to do with terrorism. Others had to do with crime and drug trafficking, industrial espionage, even environmental issues. But as long as the overall political climate was one of unconcern about national security threats, they were doomed to fall on deaf ears--and quite rightly, too, for the truth is that terrorism hadn't yet been identified as a more pressing concern than any of the other alternative top intelligence priorities that were being floated around pre-9/11.
9.12.2007 12:59pm
NickM (mail) (www):
Pointing out that failings in information-sharing and the inability of agencies to work together had been brought up repeatedly and not addressed does not suffer from hindsight bias in that these go to ANY danger to the United States.

Nick
9.12.2007 2:20pm
The River Temoc (mail):
A leader is supposed to lead, not follow public opinion.

This is well and good, but it doesn't follow that attacking countries with, say, 33% public support (and, more importantly, little institutional support from key allies) is a wise move. The whole Iraq mess should have taught us that.
9.12.2007 3:16pm
David M (mail) (www):
Trackbacked by The Thunder Run - Web Reconnaissance for 09/12/2007
A short recon of what's out there that might draw your attention, updated throughout the day...so check back often.
9.12.2007 3:27pm
Lugo:
it doesn't follow that attacking countries with, say, 33% public support (and, more importantly, little institutional support from key allies) is a wise move. The whole Iraq mess should have taught us that.

The President's job is to make the country safe and do the right thing to ensure national security, even if he is unpopular at the time. This argument is particularly unconvincing in the case of Bill Clinton, whose popularity was always over 50% during his last four years in office. "Public support" wasn't holding him back from going after the tangos in Afghanistan, that's for sure.

I have confidence that if you do the right thing in the right way - and explain it to the American people - they will support you. If Dubya did the wrong thing in the wrong way in Iraq, and failed to explain himself properly, it does not follow that Clinton could not have done the right thing with respect to al Qaeda in 1998-2000 and explained it convincingly to the people.
9.12.2007 4:13pm
cathyf:
1. Information sharing 2. The inability of the CIA, FBI, and other intel agencies to work as a unified team
But it is illegal for them to work as a team. When FISA judges refuse to sign the FBI warrants if the probable cause for the warrant was in any way tainted with NSA-gathered information, then the diligent FBI and NSA employees who diligently ensure that no NSA information taints the FBI are doing their very best to do their jobs under the law.
9.12.2007 4:16pm
Richard Aubrey (mail):
cathyf.

Yeah. I think the point is the law needs to be tweaked. It's not written in stone.
9.12.2007 4:43pm
John Burgess (mail) (www):
Lugo: When leaders say unpopular things--even if they happen to be right--they tend to get cut off at the knees for fear-mongering, for unilateralism, for not being cooperative.

BTW, nice game last night, but you really need to work on your RLISP stats.
9.12.2007 6:11pm
BGates (www):
Anderson, some of the "the curs baying for Clinton's impeachment/resignation (Repubs &Dems alike), who had no clue what was going on behind the scenes in counterterrorism," were members of Congress. Do you think Clinton have provided them with a clue, or is it only Republican executives who need to provide information to Congress?
9.12.2007 6:18pm
Morat20 (mail):
A leader is supposed to lead, not follow public opinion. The President is not a totally passive victim of public opinion, he has plenty of power to shape it. We had been attacked plenty of times by al Qaeda before Clinton left office, and if he had said "we're going to go into Afghanistan and kill those guys" - and explained why - then the public would have gotten behind it.

Were you even concious during the 90s? Clinton got impeached over a blowjob. He used cruise missiles against AQ and was accused by the very people impeaching him (Republicans) of "Wagging the Dog".

He couldn't have done jack, because the GOP Congress wouldn't have hesistated to defund any foreign adventure he cared to make.

He was stuck "uselessly lobbing cruise missiles", because he lacked Congressional support for anything else.

Congress treated Bush with kid gloves post 9/11, gave him everything he wanted -- if you think the GOP Congress form 94 onward would have allowed an invasion of Afghanistan, you should cut back on your crack usuage.
9.12.2007 6:35pm
Lugo:
Morat, that is absolute crap. Most Republicans, and in particular the leaders in the House and Senate, strongly backed Clinton's strikes on Sudan and Afghanistan, and rejected the "Wag the Dog" accusation. They would absolutely have supported stronger and more effective action against Osama, since a lot of them said the strikes were long overdue and didn't go far enough (which was of course true). It is simply absurd to argue that Clinton would not have been able to take stronger action against al Qaeda - a group that had repeatedly attacked us and that presented an ongoing threat - when he was able to take action against Serbia, a state that had never attacked us and that presented no threat whatsoever to us. Clinton could fight a 78-day war in Serbia but he couldn't launch even ONE commando raid or manned airstrike into Afghanistan? Gimme a break.
9.13.2007 12:27pm