pageok
pageok
pageok
Pitfalls of Public Opinion Polling in a Repressive Society:

This recent survey conducted in Iran claims that most Iranians believe that their authoritarian government is actually largely democratic, support the government's efforts to develop nuclear weapons, and are hostile to the United States (hat tip: Jeffrey Friedman).It is perfectly possible that these poll results really do reflect the majority opinion of the Iranian public. However, the researchers failed to consider an important factor that might bias their results: Iran is a repressive dictatorship where the government often punishes those who express dissident views that question the official line.

In such an environment, Iranians who disagree with the government's positions might hesitate to express those disagreements to foreign pollsters. They might instead toe the official line in order to protect themselves from the secret police. Obviously, good pollsters will promise respondents anonymity. But Iranians - who have lived in a police state for decades - might well distrust the sincerity of such promises. For all they know, the "pollster" who has contacted them is actually a government agent trying to ferret out dissent. Even if they believe the pollster to be sincere, there is still no guarantee that the conversation between pollster and respondent won't be overheard by the secret police.

Fear of punishment gives inhabitants of repressive societies strong incentives to lie to pollsters if they disagree with the government line. Public opinion scholars have long been aware of this problem. For example, this classic study of Nicaraguan public opinion under the Sandinista communist dictatorship showed that many Nicaraguans were only willing to express their true opinions about the government if they believed the pollsters to be affiliated with anticommunist opposition parties. In his brilliant 1995 book Private Truths, Public Lies, economist Timur Kuran documents numerous similar examples of people in repressive societies hiding their true views in order to avoid punishment by the state.

None of this proves that citizens of repressive societies never genuinely support their rulers. Adolf Hitler was genuinely popular in 1930s Germany, for example. It does, however, show that researchers should not take pro-government poll responses in such countries at face value. Scholars have understood the problem for a long time. There is no longer any legitimate excuse for polling organizations to ignore it.

I don't blame Iranians who may have lied to Western pollsters out of fear. I do blame pollsters who fail to consider the possibility that such lying may affect their results.

neurodoc:
I have my doubts about the reliability of polls that purport to show the majority of Palestinians in favor of peaceful co-existence with Israel. But those who wish to believe such results seem disinclined to any skepticism.
7.20.2007 2:45am
JBL:
I also wonder about the possibility that such polls reflect opinions that are sincere and freely expressed, but that are the result of an environment in which the available information is sharply limited or biased.

Democracy doesn't mean much without freedom of the press.
7.20.2007 3:31am
Christopher M (mail):
This is not meant to dispute the point of the post, but it's worth noting that Iran is a "dictatorship" only within a very extended meaning of the term. Certainly it's a very repressive government. But there's no one individual nor even a small, cohesive group of individuals with absolute control over the government, which I would think necessary to make a country a "dictatorship" within the usual meaning of the term.
7.20.2007 3:56am
Ilya Somin:
Certainly it's a very repressive government. But there's no one individual nor even a small, cohesive group of individuals with absolute control over the government, which I would think necessary to make a country a "dictatorship" within the usual meaning of the term.

Actually, as I understand it, most Iran experts believe that power is concentrated in the hands of the "Supreme Leader" Ali Khameini, and a few of his advisers and allies. It's a dictatorship in the same sense that, say, the USSR was a dictatorship under Brezhnev. Brezhnev didn't exercise absolute power, but was the head of a small clique (the Politburo), which collectively controlled all the levers of authority. Ditto for Iran's Supreme Leader. Such an arrangement is reasonably described as a dictatorship, as that term is colloquially understood.
7.20.2007 4:14am
Paul McMahon (mail):
There is a methodology for gathering and analyzing data that are potentially sensitive, or endangering. I first came across the idea of randomized responses decades ago, so the method is hardly new.

Essentially, what one does is have a bag with both black and white marbles, and instruct the respondent that if they draw a white marble to tell the truth (and otherwise, lie). The respondent is also told not to show the marble to the interviewer.

Now, you know the proportion of the marbles that are white (and have shown the respondents that there are equal numbers of each, by the way), so the estimation is simply a matter of probability. Note, however, that this means the sample size must be increased significantly and that in-person interviews are critical. This means that the cost will be far higher than otherwise to get the same level of precision.

I would suspect that the actual survey did not use this technique.
7.20.2007 9:09am
Richard Aubrey (mail):
Paul. Sounds like Mao's Thousand Flowers technique.
The question is not actually telling the truth. The question is admitting you know it and ARE WILLING to tell it.

I would guess many pollsters would consider the issue a feature, not a bug. After all, it gets the line they want.
7.20.2007 9:14am
Happyshooter:
As opposed to America, where we lie to pollsters due to fear of losing our jobs or being reported in the media as 'races'.

See Michigan's Constitutional Affirmative Action Ban vote, where a huge chuck of voters leaving the polling places lied to pollsters and said they opposed the ban after their major in-state employers announced that their offical policy was to oppose the ban.

In the booth they voted the way they wanted, however.
7.20.2007 9:35am
Kevin P. (mail):
JBL:

I also wonder about the possibility that such polls reflect opinions that are sincere and freely expressed, but that are the result of an environment in which the available information is sharply limited or biased.

Democracy doesn't mean much without freedom of the press.


I would go even further and say that polls are meaningless unless the press has a diversity of viewpoint and all facts and viewpoints are freely available.

I have traveled in foreign countries where the press is nominally free but toes a certain line on some issues.

Heck, in our own country, the press can be reliably counted upon to advocate some positions under the guise of providing news. Not surprisingly the opinion polls sway that way too.
7.20.2007 10:46am
jvarisco (www):
If it's so repressive, why was a reform government elected in 2000? Sounds like (gasp) actual elections. Which the conservatives - with no evidence of fraud - won in 2005. Not surprising, reall; if I'm an Iranian and Bush has just invaded Iraq which had no WMDs and presented no threat, but was merely talking to North Korea, I'd want nukes to protect myself too.
7.20.2007 11:33am
Justin (mail):
Ilya, I know enough Iran experts to know that what you said, given the qualification that you said it, (i.e., "most Iran experts believe"), is hogwash. Obviously, that doesn't mean most experts believe Khameini isn't powerful, but a Brezhev-styled dictator?

Now, I'm not an expert, and the underlying statement may be true. But you should not rely on an argument of authority in this instance, when you don't seem to understand the complexity that such authority gives to the question at hand.

All the Iranian experts I know think Iran, while nowhere near a pure democracy, is the second most democratic state in the Middle East (next to Israel), and whose democracy functions better than many other mixed systems, such as China, Vietnam, most African nations, and (depending on the expert) even some Latin American nations.
7.20.2007 11:36am
neurodoc:
jvarisco:

Given my druthers, I would prefer to think that the Iranian electorate did not "freely" and "informedly" choose these governments. Given your druthers, you would prefer to think that they did, or you are just expressing what you consider to be a "realist" point of view? Same thinking about the Palestinian elections that saw Hamas installed?

"...Iraq which had no WMDs and presented no threat..." Are you speaking with that certainty from a post-2003 perspective or saying there was no doubt about those matters pre-2003? You would agree that without a doubt Iraq had WMD in 1991 and had demonstrated a willingness to use them, wouldn't you?
7.20.2007 11:53am
Ilya Somin:
If it's so repressive, why was a reform government elected in 2000? Sounds like (gasp) actual elections. Which the conservatives - with no evidence of fraud - won in 2005.

3 points:

1. In both cases, the candidates elected had to be preapproved by Iran's clerical authorities. Real reformists (those actually in favor of abolishing the existing system and establishing a true democracy were not allowed to run).

2. The positions up for "election" were not those that had real power (e.g. - the post of Supreme Leader, which is not an elected position).

3. As other commentators have pointed out, there was no free press, or right of true opponents of the goverment to put their views forward.
7.20.2007 12:41pm
Ilya Somin:
All the Iranian experts I know think Iran, while nowhere near a pure democracy, is the second most democratic state in the Middle East (next to Israel), and whose democracy functions better than many other mixed systems, such as China, Vietnam, most African nations, and (depending on the expert) even some Latin American nations.

3 points here too:

1. Iran is surely less democratic than Turkey (which has true contested elections), Lebanon, Kuwait, and several other Middle Eastern states.

2. The other states you compare it too (China, Vietnam, etc.) are not democratic at all (having no opposition parties that are allowed to run elections).

3. The real power in Iran is in fact in the hands of the Supreme Leader and his clerical allies and advisers, who have the right to veto any decisions by the "elected" president and other similar officials. Perhaps this is more "democratic" than, say, Saudi Arabia, but it's not much of a distinction.
7.20.2007 12:44pm
markm (mail):
Turkish democracy is limited in that the military stands ready to overturn elections, if in their opinion the winners are a threat to democracy (e.g., radical Islamists). It sounds weird, but so far it's worked, and it makes Turkey the second-most democratic nation in the middle east, after Israel.

Note that democracy is limited in the USA too - the rights of political minorities are protected.
7.20.2007 1:17pm
Justin (mail):
Ilya, I'm not sure why you use the word surely, and I can't tell if you've done research on the topic or are simply relying on popular misconceptions. Kuwait, for instance, is a constitutional monarchy - up until 2005, when women were granted the vote, only 15% of the citizenship had the right to vote - but the Prime Minister and Emir both remain, practically, a hereditary position.

http://en.wikipedia.org/wiki/Kuwait
7.20.2007 1:57pm
ys:

All the Iranian experts I know think Iran, while nowhere near a pure democracy, is the second most democratic state in the Middle East (next to Israel), and whose democracy functions better than many other mixed systems, such as China, Vietnam, most African nations, and (depending on the expert) even some Latin American nations.

May I remind you about the staged TV "confessions" of Iranian-Americans imprisoned in the Islamic Republic. Took place just a couple of days ago and the main words pounced on by the captors were "...networks formed...", as in conspiracies hatched at academic conferences. This was also the favorite terminology of forced confessions in 1937 under Stalin (I read up on their methodology lately). I am not saying Khamenei is already a Stalin, but he is a wannabe.
7.20.2007 4:39pm
Ilya Somin:
Ilya, I'm not sure why you use the word surely, and I can't tell if you've done research on the topic or are simply relying on popular misconceptions. Kuwait, for instance, is a constitutional monarchy - up until 2005, when women were granted the vote, only 15% of the citizenship had the right to vote - but the Prime Minister and Emir both remain, practically, a hereditary position.

I was referring to Kuwait today, not Kuwait pre-2005. But even pre-2005, the 15% of the population of Kuwait who could vote could at lest vote for a legislature with real power and for candidates not preselected by the government. Neither is true in Iran.
7.20.2007 6:19pm
Can't find a good name:
Despite Iraq's numerous problems, it should also be on the list of countries in the Middle East more democratic than Iran.
7.21.2007 1:07am
Justin (mail):
Ilya,

I don't think you are trying to be rational about this, and am no longer interesting in discussing this. At no point do you do anything other than make what seems like absurd ascertions based only on truthiness, nor do you seem to give even an inkling of the support you have for any of your positions. Please let me know if you decide you want to explain your reasoning re: Iran in terms of actual facts.
7.21.2007 2:14am
David M. Nieporent (www):
Justin:

I don't think you are trying to be rational about this. Please let me know if you decide you want to explain your reasoning re: Iran in terms of actual facts. You don't have to pretend you know "Iran experts," Justin. Iran has, apparently, reasonably fraud-free elections. That is not the definition of democracy. "Reformers" regularly have supportive media outlets shut down, those elections are not for the positions of real power, and are only among the candidates allowed by those who do have real power. That isn't a democracy. If you know "Iran experts" who think it is, you ought to ask for a refund.
7.21.2007 3:01am
Can't find a good name:
I can't read Chinese, but somehow I suspect that Aizheng's comment may not be on topic, given that almost every single word is a hyperlink.
7.21.2007 12:29pm
neurodoc:
David Nieporent, do you really expect Justin to return here and engage with you in rational discourse about these matters, or indeed about much else? When Professor Somin informed him that in democracies, even quasi-democracies, there are no Supreme Leaders vetting would-be candidates for public office, Justin couldn't handle it and stalked off in a snit. ("I don't think you are trying to be rational about this, and am no longer interesting in discussing this...Please let me know if you decide you want to explain your reasoning re: Iran in terms of actual facts.") Not very likely that he is going to return to answer you about the shutting down of opposition newspapers or various other authoritarian manifestations of the Islamic Republic of Iran? (BTW, in how many "democracies" do agents of the state go about terrorizing unmarried couples, sometimes even murdering them without fear of any consequences?) If you insist on messing with Justin's weltanshauung, you will only agitate the fellow further, and you may upset his anti-war, anti-Israel Jewish GF or ex-GF, the one on the No-Fly list, too.
7.21.2007 10:09pm