A recent ABC-BBC-ARD survey of Afghanistan has both given hopes to those who support NATO’s mission in Afghanistan and created a backlash among doubters.

ABC News polling director Gary Langer begins his summary of the findings with the good news:

Hopes for a brighter future have soared in Afghanistan, bolstered by a broad rally in support for the country’s re-elected president, improved development efforts and economic gains. Blame on the United States and NATO for violence has eased – but their overall ratings remain weak.

In one key shift, the latest poll by ABC News, the BBC and ARD German TV finds that sharply more Afghans now see the Taliban as the main source of their country’s strife, while many fewer blame the United States or its allies – significant progress in a central aim of the new commander of U.S. and NATO forces, Gen. Stanley McChrystal.

Another, basic change is larger still: After steep declines in recent years there’s been a 30-point advance in views that the country is headed in the right direction; 70 percent now say so, the most since 2005. Afghans’ expectations that their own lives will be better a year from now have jumped by 20 points, to 71 percent, a new high. And there’s been a 14-point rise in expectations that the next generation will have a better life, to 61 percent.

New America Foundation president Steve Coll enthuses, "Afghans seem to be more optimistic about Obama’s policy toward their country than Americans are. Since their decisions will be more decisive than ours in the outcome, that has to be counted as a good thing."

BBC world affairs correspondent Adam Mynott notes mixed news for NATO.  On the one hand, "For Nato troops, including UK forces, support has risen from 59% to 62%."  On the other,

Most Afghans appeared positive about the presence of troops from Nato and other countries stationed in Afghanistan.

The survey also asked if people thought it was good or bad that US forces entered Afghanistan in 2001 to drive out the Taliban. Of those questioned, 83% said it was either very good or mostly good. This compares with 69% for 2009.

However, more of those questioned believe troops with the International Security Assistance Force (Isaf) are now worse at avoiding civilian casualties (43% worse and 24% better).

There was some ambivalence about how long Isaf forces should remain in the country – 22% said they should leave within the next 18 months, and 21% said they should stay longer than 18 months from now.

The Alliance, naturally, is taking a glass half full approach.

NATO is encouraged by a recent poll on Afghanistan that indicates greater popularity for NATO’s mission and decreased support for the Taliban, the alliance said in a statement on Monday.

[…]

"The increasing optimism of the Afghan people in their future is welcome news to NATO and the men and women of the International Security Assistance Force (ISAF). Our population centric strategy and the commitment to add 37,000 more troops in 2010 is creating a new momentum for the mission, which is reflected in this poll", said the NATO Spokesman, James Appathurai.

But, how seriously should we take the survey results?   Many analysts believe it impossible to do accurate public opinion sampling in authoritarian countries — let alone those in the midst of a brutal civil war.

As Sameer Lalwani noted about the election polling back in August,

First, polling will suffer from tremendous sampling bias, in which the sample is not representative of the national population. Afghanistan is still one of the poorest countries in the world with an extremely low telecommunications penetration rate. According to the International Telecommunication Union, there is less than one fixed phone line per thousand people. The world average is about nineteen per thousand.

Afghan mobile phone usage has increased exponentially in recent years to 29 per thousand but this still remains half the global average. This telecom divide means phone polling tends to favor wealthier urban constituents and under-sample rural areas. Pollsters can correct for this with face-to-face interviews (as the IRI and Glevum Associates did in their recent election polls) though they are more time consuming and expensive. But polling in a conflict-zone incurs new sampling biases by tending to over-sample safer and less-conflict ridden areas.

Second, polling in America is far different from polling in Afghanistan. A colleague who extensively traveled and studies the Middle East once warned me about putting much stock in foreign polling numbers, particularly out of American firms. His reason was this: the United States is an open society with a long history of polling and consumer surveys that acculturates people to be more responsive and forthright in their opinions. In conflict-ridden societies with a history of repressive governments, people have learned to guard their beliefs and are more likely to mislead polling interviewers out of fear retribution from the central government, local elites, or insurgents.

Huffington Post‘s Ryan Grim interviewed five "people who have worked extensively on the ground in Afghanistan" — only one of whom, it should be noted, has any polling expertise — and they say it’s "impossible" to get accurate results.

The Taliban controls large swaths of the country and the war has made much of the country unsafe to travel through. The Taliban doesn’t do surveys, so anybody approached by somebody with a clipboard knows that the person either represents foreign troops, the central government or a private company associated with one or both.

Then there are the not-so-immediately obvious reasons: Afghanistan is a highly patriarchal society, meaning that getting a woman’s true opinion is extremely hard. [Prakhar Sharma, head of research at the Centre for Conflict and Peace Studies (CAPS) in Kabu] said that his research teams have never been able to get even close to the 50-50 male/female split that the ABC survey claims.

Getting a man’s honest opinion is no simple task, either, he said, because the responses are calculated to protect and benefit the respondent’s family and village. "The Afghans know it when they see sudden changes in development assistance, changes in government officers, police tashkils/numbers [the Tashkils are the national police], more/less operations immediately after the polls. It is difficult to pretend to them that the polls do not matter. Their responses are therefore calculated," he said.

While the other interviewees have some strong thoughts as well, their expertise on polling methodology is limited.  But Sharma’s basic complaint has long been the conventional wisdom:  One simply can’t poll in societies where people don’t feel free to speak their mind.  And, while Afghanistan is ostensibly a democracy, the widespread presence of Taliban thugs presents, to say the least, an enormous chilling effect on free expression.

CSM’s Ben Arnoldy notes some skepticism within the region, too.

It’s morning in Afghanistan, according to a new nationwide poll of Afghan public opinion. But while even some skeptics agree that Afghans may have a more optimistic view of their country than outsiders suppose, analysts with deep experience there are cautious about the sunny picture emerging from the survey released Monday, saying it runs counter to their sense of the mood and the repercussions of the recent fraud-riddled election.

[…]

“We Afghans sometimes don’t know if the person questioning is someone from the government or whether it’s honest polling,” says Haroun Mir, director of Afghanistan’s Center for Research and Policy Studies based in Kabul. Unfamiliarity with polling can also bump up against cultural traits: “Answers from Afghans are not one word, sometimes it’s a 10-minute speech. And then it’s up to those conducting the polling to get one answer from it.”

But Langer provides a cogent rebuttal.  His response is point-by-point and rather longish but here’s an excerpt:

Our latest poll is our fifth in Afghanistan since 2005, conducted with the same high level of methodological standards as those that came before. Like our polling in Iraq, we see these surveys as an essential source of independent information on conditions on the ground, unfiltered by government or military authorities, or, frankly, by the perceptions of others with an ax to grind or a viewpoint to promote. We conduct these polls to obtain directly the views of Afghans themselves, and we believe our knowledge would be vastly poorer without them.

[…]

Field work for our Afghanistan surveys is managed by ACSOR, the Afghan Center for Socio-Economic and Opinion Research, a subsidiary of D3 Systems in Vienna, Va. ACSOR maintains staffs of trained interviewers in each of Afghanistan’s 34 provinces; they travel in separate male and female teams to randomly selected sampling points, from there follow a random route to select households and then randomly select a resident to interview.

[…]

On Sharma’s suggestion that respondents provide the answers interviewers want, for this to be so the interviewers have to telegraph a preferred response. Avoiding bias can be a challenge for Westerners in Afghanistan. Our local Afghan interviewers are trained to conduct their research neutrally. And they’re not told the identity of the sponsor of the survey they’re carrying out.

If respondents colored their responses to benefit their village, presumably they’d have tried to draw greater assistance by saying things are in terrible shape and getting worse. In fact we see better ratings of local conditions. Larger numbers of Afghans report roads and clinics built or repaired and electricity supplied. Forty percent give a positive rating to the availability of jobs and economic opportunities – still far from ideal, but up from 26 percent in 2007.

Better development is one of several factors that have improved the outlook of ordinary Afghans for their own and their country’s future. While there’s been progress in some areas, there are significant challenges ahead. We can understand them best not through the representations of officialdom, nor through punditry, conventional wisdom and anecdote, but through independent, reliable and direct measures of the Afghan public’s attitudes and experiences. That’s why we poll there.

I spoke with Jim Burton, a vice president at Public Opinion Strategies* who has fielded surveys and taught polling methodology in Iraq.  He notes that, in recent years, many NGOs and other reputable organizations have done polling in very hostile countries with significant success.  The key is knowing "what the expectations are" when going in and using "the best methodology possible" under difficult circumstances.

In Afghanistan, for example, the last census is nearly 15 years old so proper weighting of the sample to match national demographics — as is routinely done in surveys of Western countries — is not possible.   So researchers have to be realistic about such things and understand that the results are "not going to be perfect."

Burton notes that Langer is "a stickler on methodology" and "deserves credit" for his efforts over the years in Iraq and Afghanistan.  So, while we may not want to assign the same level of precision to the numbers in this Afghanistan survey as we would one conducted in the United States, England, or Germany (the host countries for the news organizations behind the work) we can nonetheless take them quite seriously as the best estimates possible under extremely arduous circumstances.

James Joyner is managing editor of the Atlantic Council.  Top graphic: ABC News.  Lower graphic:  BBC.

*In the interests of full disclosure, my wife is COO of Public Opinion Strategies and Burton is a longtime family friend.  His expertise in this matter is unimpeachable, even moreso in that he’s praising the work of a competitor.

Image: afghanistan-where-things-stand.jpg