pageok
pageok
pageok
Wessel on Frank on Economists:

A quick post script to my previous post on David Wessel's economics column in today's WSJ concerning the stimulus. Wessel quotes Barney Frank in passing:

Not for the first time, as an elected official, I envy economists. Economists have available to them, in an analytical approach, the counterfactual. Economists can explain that a given decision was the best one that could be made, because they can show what would have happened in the counterfactual situation. They can contrast what happened to what would have happened. No one has ever gotten reelected where the bumper sticker said, "It would have been worse without me." You probably can get tenure with that. But you can't win office.

I didn't include this in my discussion, as I found it a bit tangential. However, when I saw that Professor Mankiw had linked to it, I decided to revisit it.

Leave aside as too ad hominem whether Congressman Frank's own statements regarding his role in Fannie and Freddie have any element of counterfactual. I understand that what some people, certainly in his Congressional district, find clever, I find too clever by half, but okay - that is really not my interest here, and please, not in the comments.

Is it really true, as a general proposition, that politicians have no recourse to counterfactuals? What about, for example, the famous claim of "saved or created" four million jobs? As accounts of the President's February 9, 2009 press conference had it:

Question: The American people have seen hundreds of billions of dollars spent already, and still the economy continues to free-fall. Beyond avoiding the national catastrophe that you've warned about, once all the legs of your stool are in place, how can the American people gauge whether or not your programs are working? Can they — should they be looking at the metric of the stock market, home foreclosures, unemployment? What metric should they use? When? And how will they know if it's working, or whether or not we need to go to a plan B?

Answer: I think my initial measure of success is creating or saving 4 million jobs. That's bottom line No. 1, because if people are working, then they've got enough confidence to make purchases, to make investments. Businesses start seeing that consumers are out there with a little more confidence, and they start making investments, which means they start hiring workers. So step No. 1, job creation.

Jobs "created" is a factual. Jobs "saved," on the other hand, poses a counterfactual. Professor Mankiw has an excellent quick discussion of why:

The expression "create or save," which has been used regularly by the President and his economic team, is an act of political genius. You can measure how many jobs are created between two points in time. But there is no way to measure how many jobs are saved. Even if things get much, much worse, the President can say that there would have been 4 million fewer jobs without the stimulus.

An actual answer to the question "What metric?" could have taken the form: "If the unemployment rate on [insert date] is below [insert threshold], I will judge the plan to be a success." Given the uncertainties inherent in the economy, however, no sensible politician would hold himself to such a measurable standard. But the President also wanted to avoid sounding like he was avoiding accountability. So he gave us a non-measurable metric. A clear and specific benchmark, without any way of ever knowing whether it has been reached.

A completely honest (but perhaps politically ill-advised) response to the question would have been, "Geez. I am only President of the United States. I cannot be held responsible for everything that what happens with the economy!" If he had said that, I would have agreed with him.

---- Update: A regular reader of this blog (who deserves anonymity) misinterpreted my meaning, so let me clarify: The 4 million job number is a counterfactual policy simulation of what the stimulus will do based on a particular model of the economy. As such, I have no objection to someone citing it in a policy discussion. In fact, macroeconomists use models to generate figures like this all the time. I have even done it myself.

But as an answer to the question "how can the American people gauge whether or not your programs are working?... What metric should they use?", citing the 4 million job figure is a non sequitur, or more likely a diversion. A metric has to be measurable, and the actual number of jobs "created or saved" by the policy will never be measurable from any data source.

The more I think about it, the more I think, contra Frank, that politicians of all ideological stripes get elected on counterfactuals all the time. Am I right about that - not about Barney Frank, please - or not? Other examples of politicians offering counterfactuals? And are counterfactual arguments always bad, while we're taking up the abstract category? When and how, or not?

Update: While I am at it, let me ask what the abstract nature of counterfactual argument is. What makes a counterfactual argument a counterfactual? I usually think of them as "but for" arguments - but maybe that is too quick. What is the abstract structure of a counterfactual? And what, in that case, makes for a good as opposed to bad counterfactual argument? Finally, does this bear any relationship to argument from analogy, and if so, how?

Dan Schmutter:
How about the claim that "our policy of X has prevented another terrorist attack in the U.S."

Dan Schmutter
8.20.2009 5:42pm
rick.felt:
The expression "create or save," which has been used regularly by the President and his economic team, is an act of political genius. You can measure how many jobs are created between two points in time. But there is no way to measure how many jobs are saved. Even if things get much, much worse, the President can say that there would have been 4 million fewer jobs without the stimulus.

Isn't there still a counterfactual when you discuss how many jobs were "created"? Let's say that by 2012 the economy has 4 million more jobs than when Obama took office. Sure, we can "measure" that there are more jobs, but we can't really measure how many of those jobs wouldn't be there had Obama done nothing, or had Obama done what his opponent suggested.

Whether you're estimating the number of jobs "saved" or the number of jobs "created," you're still speculating as to the number of jobs influenced by the President's actions.
8.20.2009 5:46pm
Kenneth Anderson:
Quite right - in order to make it a factual, one would have to know that, but for our policy, there would have been an attack.

Let me just be clear about the general issue - it's not that I'm opposed to counterfactual arguments. On the contrary, we can't do real policy without them. But I don't think it's the case that politicians cannot get elected making them, whether defensible counterfactual propositions or not.
8.20.2009 5:47pm
Leo Marvin (mail):

How about the claim that "our policy of X has prevented another terrorist attack in the U.S."

... and its corollary, "fighting them over there so we won't have to fight them here."
8.20.2009 5:48pm
Steve:
"Jobs created" is hardly a factual. You can say "look, here's a new job," just like you can say, "look, this person still has a job," but what's debatable in both cases is the element of causation. Maybe those jobs would have come into existence anyway even without the stimulus. Maybe that job would have remained in existence even without the stimulus. Both arguments are debatable and ultimately both are unprovable.

Whenever good things happen (like jobs being created), supporters of the President tend to argue that they happened because of the President's policies, and opponents tend to argue that they would have happened anyway. It's probably not coincidence that the arguments tend to play out this way. But even if ten million jobs came into existence between now and next year, no one would have to concede that they were "created" by the stimulus; they could still argue that the stimulus was completely ineffective. So I don't see "jobs created" as any more of a hard fact than "jobs saved."
8.20.2009 5:49pm
egd:
While the "or saved" part sounds like complete baloney, the "created" part may be misleading as well. Real recovery in the jobs market requires two parts: 1) increasing the number of jobs created; and 2) reducing the amount of jobs lost.

The first part represents the "created" portion of the Obama plan while the second represents the "saved" portion. Only when jobs are created AND saved (or created in excess of those not-saved) is actual job growth, and therefore perceived economic recovery, realized. The big deception of Obama's statement is the word "or".

The question of "How many jobs are created" is just as factually untenable as "how many jobs are saved," at least when your sole metric is unemployment rate.

We can measure the number of jobs saved by speaking directly to employers and asking them how many jobs they planned on shedding in the '10 fiscal year before the stimulus package. Then compare that to the number of actual jobs each employer shed in the '10 fiscal year. It represents a reasonable measure of the number of jobs "created or saved" we can attribute to the stimulus package.

Therefore, the rationale for "jobs created" or "jobs saved" is directly related to actual, measurable experiences.

That said, the fact that unemployment exceeded the administration's expectations should rationally lead to the conclusion that rather than "creating or saving" jobs, the stimulus has actually reduced the number of jobs.

Long story short, I disagree with the idea that jobs "created or saved" cannot be measured. Comparing projected unemployment with actual unemployment gives an excellent metric by which to measure "created or saved" jobs.

It just turns out that the Obama administration has failed to do either.
8.20.2009 5:56pm
geokstr (mail):
The Clintonistas still claim that Bill's 1993 tax increase, the largest in history until very recently, was the cause of the booming economy and balanced budget during his two terms. It would be interesting to see a counterfactual study as to what the economy might have been like if he hadn't slowed it down with the tax increase.
8.20.2009 6:02pm
erp:
Counterfactual = lie.
8.20.2009 6:05pm
Ex parte McCardle:
To answer the questions in your update, investigate the classic study, one of the wonders of analytic philosophy of the past 50 years, David Lewis's Counterfactuals (Basil Blackwell, 1973), but be forewarned that it is not exactly casual reading.
8.20.2009 6:06pm
Kenneth Anderson:
Ex Parte McCardle - excellent suggestion, thank you.
8.20.2009 6:09pm
josil (mail):
Re the "terrorist" example, the counterfactual works both ways. Libertarians have been known to claim we would not have been attacked if we avoided foreign entanglements (e.g., "by staying out of the Middle East").
8.20.2009 6:12pm
Kenneth Anderson:
I'm somewhat surprised, as a business law professor, that no one has mentioned counterfactuals in relation to opportunity cost or avoided cost arguments.
8.20.2009 6:21pm
sk (mail):
Note that it is frankly impossible to function without counterfactuals (or, in scientific/engineering speak, alternatives not selected). The essence of the scientific decisionmaking process is to
1) ID a problem
2) gather information
3) propose solutions
4) compare and contrast proposed solutions
5) make a selection.

This process is used in science, engineering, consulting, and military decisionmaking.

Counterfactuals are all the unselected alternatives.

Good counterfactuals are plausible unselected alternatives.
Bad counterfactuals are implausible unselected alternatives (in the speak of the above process, perhaps a proposed solution that was determined to be unrealistic, or a solution that wasn't even considered-and thus unstudied).

the problem with the political process is it doesn't follow the scientific decisionmaking process*, and thus multiple solutions aren't studied, or its a complicated enough problems that the the consequences of solutions are impossible to predict. Thus, counterfactuals are alternative solutions to a problem that hasn't been adequately defined, or solutions that haven't been compared or contrasted, or for which information wasn't gathered. thus, it becomes an information-poor 'sense' of whether a counterfactual (an alternative solution) is reasonable or not.

Note that the above formalized decisionmaking process, I bet, was created by some psychologist in the 60's (perhaps to improve business decisions, perhaps to define the scientific method, I don't know). I have encountered it in the military (the Military Decisionmaking Process, or MDMP), as well as in engineering and governmental policy. If you research it, you will probably find one source for it sometime within the last 40 years.

Sk

*I'm not saying the political process should be a scientific process. I'm saying the problem with identifying plausible conterfactuals in the political process is due tot he fact that its not a scientific process.
8.20.2009 6:40pm
byomtov (mail):
I'm somewhat surprised, as a business law professor, that no one has mentioned counterfactuals in relation to opportunity cost or avoided cost arguments.

The counterfactual in those cases is often relatively easy to support.

"If we hadn't expanded the factory we would not have been able to meet the surprisingly strong demand for our new widgets."

"If we hadn't introduced New Coke we would have saved [some huge amount] and avoided a major embarrassment."
8.20.2009 7:02pm
Steve:
the problem with the political process is it doesn't follow the scientific decisionmaking process*, and thus multiple solutions aren't studied, or its a complicated enough problems that the the consequences of solutions are impossible to predict.

The problem is that while economics does its best to predict things, on the macro level you can't really run the same experiment twice the way you could in the laboratory.
8.20.2009 7:04pm
Leo Marvin (mail):

But I don't think it's the case that politicians cannot get elected making them, whether defensible counterfactual propositions or not.

I agree, but only if the voters are afraid the counterfactual may still happen, e.g., a terrorist attack. The economic apocalypse we avoided isn't a pressing concern any more, making Obama a victim of his success. Our pre-occupation with economic matters is more like what happens in all cyclical recessions, albeit a very bad one. And that raises a second factor: how clear is the contrast between current conditions and what could have been? For Bush, the contrast between 9/11 and no new 9/11's was clear, even if part of his "cure," the Iraq war, was worse than the disease. For Obama, the contrast is murkier for a couple of reasons: first, we don't have any fresh memory of an economic collapse comparable to what we just avoided; second, for people who lost their jobs, businesses, houses and other investments, the recession we got may seem indistinguishable from the depression we skirted. It would be hard for many of them to believe they're better off than they could have been. It's like a doctor telling an accident victim he saved his life but had to amputate the legs. The patient may appreciate what the doctor did for him someday, but probably not in this election cycle.
8.20.2009 7:08pm
Andy Bolen (mail):
You have lots of good posts; this is a great post.
8.20.2009 7:21pm
Recovering Law Grad:
I don't agree that it is impossible to estimate the number of "jobs saved." In fact, it may be easier than determining the number of jobs created.

This is because a great deal of the job savings occurred as the result of states forgoing budget cuts that, had they gone through, would have resulted in state gov't workers losing their jobs. (Of course, many conservatives don't consider state jobs actual "jobs," or, alternatively, value them highly, but that's another story.)

We can figure out how many jobs were saved by (1) examining state budget proposals brought up before the stimulus passed and the number of cuts they called for, or (2) in the absence of specific budget proposals, estimating how many state jobs would have been lost had states passed non-stimulus-supported budgets. The latter may not be that difficult, since most states have balanced budget amendments.

I realize that this would not be perfect, but the causal link between avoiding gov't layoffs and stimulus money seems stronger than the causal link between the stimulus and the creation of private sector jobs.
8.20.2009 7:27pm
ShelbyC:

I don't agree that it is impossible to estimate the number of "jobs saved." In fact, it may be easier than determining the number of jobs created.


It's not. It's not. It's impossible to credibly estimate the number of "jobs saved" when your butt is on the line based on that number. So if I go to my boss and say, "look, the department is operating 10% more efficiently since you hired me, so I saved you alot of money", he might or might not believe that the 10% is because of me, but if I go to him and say, "well, I've estimated that the department would have operated 10% less efficent if you hadn't hired me, but we're operating at the same level, so I saved you money" he's going to think I'm a weasel.
8.20.2009 7:40pm
Malvolio:
Libertarians have been known to claim we would not have been attacked if we avoided foreign entanglements (e.g., "by staying out of the Middle East").
Those aren't libertarians; they're just stupid. They might also be libertarians, but that remark is made possible by stupidity, however motivated.

When President Obama claims to "saved or created" X number of jobs, he does so in calm confidence that there is no parallel US, identical except that his policies are not applied, against which we could compare our employment figures -- his statement is therefore not falsifiable. It may be true, it may be false, we'll never know.

By contrast, someone who claims that if a country "stays out of the Middle East", it would be safe from terrorism, apparently believes that either no country has ever stayed out of the Middle East or that every country that has done so has been spared terrorism. (Spain? Germany? Indonesia? Sri Lanka? India?)
8.20.2009 7:49pm
SenatorX (mail):
99.9% of politicians are lying weasels (and Barny is near the top of the list). The counterfactual argument is just a device that allows them to lie and not be caught, therefore perfect. They can't use plausible deniability all the time so they have to mix it up with some counterfactual arguments. Both stink to the discriminating American but their votes don't outweigh the useful idiot masses. So the weasels are trained to keep lying and the game goes on. I predict that because they are so useful to politicians and government is growing so much that the next bubble may be in counterfactual arguments. To bad there isn't an ETF for that.
8.20.2009 8:17pm
Bruce Hayden (mail):
How about the claim that "our policy of X has prevented another terrorist attack in the U.S."
You are looking at this as someone without (most likely) the security clearance required to see the effects of the policy.

What the government can say is two fold. First, and maybe less helpful, is that there haven't been any major terrorists attacks since 9/11/01. But that may be just because the terrorists shot their wad then.

But they can also say that Y attacks were thwarted, and likely also that Z policy allowed them to thwart them. How? Let us take the TSP (Terrorist Surveillance Program), where international calls were sometimes tapped w/o a warrant. They may be able to say, well, we got 100 suspicious calls, investigated them, found 20 suspicious people engaging in terrorist activities, and arrested them. Or maybe they caught an arms shipment as a result. That sort of thing. Identifiable instances of the policy resulting in a terrorist attack being foiled.

But then, coming back to the first situation, about there not being any major terrorist attacks since 9/11/01, when combined with thwarting Y attacks using Z policy, maybe one can say that the lack of major terrorist attacks was at least partially due to using Z policy.

Another key difference here though is that since there have been no major terrorist attacks since 9/11/1, there is no countrafactual. There is no possible examples of other attacks succeeding because of the policy, while others were thwarted.
8.20.2009 8:19pm
SenatorX (mail):
8.20.2009 8:31pm
RPT (mail):
But for Tom Ridge's bogus "terror alerts", John Kerry would have been elected in 2004.
8.20.2009 8:59pm
Kenneth Anderson:
Senator X: negative interest might be a foolish policy - or not - and Greg Mankiw might be many things, but "fool" is not among them.

But I've been down this path before, when I rather gently chastised Ezra Klein on this blog for talking about the 'incredible lightness' of Greg Mankiw, and took untold grief for it in the blogosphere for challenging the purveyor of Journolist.
8.20.2009 9:06pm
Jiffy:
Is Mankiw really arguing that policy can be legitimately evaluated only by comparing conditions at the time the policy was implemented with those at some later time and that it's illegitimate to consider what would have happened absent implementation of the policy? So if I jump from an airplane, pull my chute, hit the ground hard and break my leg, pulling the chute was a bad idea because it caused me to break my leg?
8.20.2009 9:13pm
LarryA (mail) (www):
But as an answer to the question "how can the American people gauge whether or not your programs are working?... What metric should they use?", citing the 4 million job figure is a non sequitur, or more likely a diversion. A metric has to be measurable, and the actual number of jobs "created or saved" by the policy will never be measurable from any data source.
On a national level, probably not. However, on an individual level first I lost my job due to funding cuts by a not-for-profit. Second, this (six months) is the longest I've ever been substantially unemployed (except for my moonlighting business and a couple of short-term jobs) without even a close encounter with being hired. Third, I submitted a resume today for a position, and the person in charge told me that they advertised the same job two years ago and got four resumes total. This time, early in the hiring process, they already have twenty-two.

If you have a job, hang on to it.
8.20.2009 10:08pm
SenatorX (mail):
I guess it's all a matter of perspective. If you are a Keynesian who things permanent inflation is a great thing (as most elites with assets close to the teat do) then you might think Mankiw's ideas are great. Others might look at the history of the federal reserve's spoken mandate of "price stability" and what's happened to the dollar over the past 100 years with a bit of a sneer. Blowing a constant bubble until loose credit eventually outpaces the ability to service even the interest on debt and then responding to that inevitable deflationary episode by trying to blow a bigger bubble, is ...

I don't care about Mankiw, there are many like him. What irritates me on this site and I guess will continue to irritate me is that supposed libertarians support the federal reserve. There is nothing about the federal reserve in its creation, track record, or current machinations that deserves kudos from a libertarian. At best you are stuck with the discordant "Ignoring what got us into this mess(the fed), THEY are best suited to leading us out by doing what they did before. But MORE OF IT!!!" Whatever.
8.20.2009 10:16pm
Ohgoodgrief (mail):
The art of the politician is similar, I believe, to the used car salesman, the gritter and the other guys who sell dreams. It's fairly easy to read the desires of the poor soul who needs a car, new siding on his house or who craves political certainty. So people like Frank provide just enough information to lead the politically unsavvy to the conclusion they envision. Meanwhile, the politician provides just enough gratification to keep his constituents on the hook.

Change we can believe in!
8.20.2009 10:47pm
cs3:
The way the modern practicing empirical economist (as opposed to a philosopher, say) views counterfactuals and the issue of casual inference can be pretty well summarized in a couple old articles:

Donald Rubin, "Estimating Causal Effects of Treatments in Randomized and Nonrandomized Studes", J Educ Psych 1974

This is a non-mathematical discussion of the idea of counterfactuals and experimentally or observationally determining causal effects from counterfactuals.

Paul Holland, "Statistics and Causal Inference", J Am Stat Assoc 1986

A little more statistical, with some elementary probability theory built in, but also more wide coverage of various approaches to understanding counterfactuals and causality.
8.21.2009 12:28am
Kenneth Anderson:
cs3 thanks, those are helpful background sources.
8.21.2009 12:42am

Post as: [Register] [Log In]

Account:
Password:
Remember info?

If you have a comment about spelling, typos, or format errors, please e-mail the poster directly rather than posting a comment.

Comment Policy: We reserve the right to edit or delete comments, and in extreme cases to ban commenters, at our discretion. Comments must be relevant and civil (and, especially, free of name-calling). We think of comment threads like dinner parties at our homes. If you make the party unpleasant for us or for others, we'd rather you went elsewhere. We're happy to see a wide range of viewpoints, but we want all of them to be expressed as politely as possible.

We realize that such a comment policy can never be evenly enforced, because we can't possibly monitor every comment equally well. Hundreds of comments are posted every day here, and we don't read them all. Those we read, we read with different degrees of attention, and in different moods. We try to be fair, but we make no promises.

And remember, it's a big Internet. If you think we were mistaken in removing your post (or, in extreme cases, in removing you) -- or if you prefer a more free-for-all approach -- there are surely plenty of ways you can still get your views out.