Archive for April, 2011
If you haven’t been following it, there’s currently a lot of controversy swirling around Greg Mortenson, co-author of Three Cups of Tea and co-founder of the Central Asia Institute. On Sunday 60 minutes aired accusations that Mortenson fabricated the ‘creation myth’ of the organization, a story about being kidnapped by the Taliban, and more. The blog Good Intentions Are Not Enough is compiling posts related to the emerging scandal, and the list is growing fast.
If you haven’t read it already, Jon Krakauer’s mini-book, Three Cups of Deceit: How Greg Mortenson, Humanitarian Hero, Lost His Way, is really worth the read. Completely engrossing. It’s a free download at byliner.com until April 20. It’s about 90 pages, and Krakauer has obviously been researching it for a while — in fact, my guess is that Krakauer turned 60 Minutes onto the story, rather than vice versa, which would help explain why he was featured so heavily in their piece. In the TV interview Krakauer quotes several former employees saying quite unflattering things about how CAI is run, so it’s good to see that he gets many of those people on record in his ebook.
A few disclaimers: I think it’s worth pointing out that a) as a one-time supporter and donor to CAI, Krakauer arguably has an axe to grind, b) several of Krakauer’s previous books (Into Thin Air, Into the Wild, and Under the Banner of Heaven) have had sections disputed factually, though to me Into the Wild is the only case where he seems to have actually gotten things wrong, and c) I’m a big fan of him as a writer and thus am possibly a bit predisposed to believe him. Admitting by biases up front like good epidemiologist.
That said, it sounds like CAI has been very poorly led. Krakauer’s book levels many damning claims about Mortenson and CIA’s financial management that, while less emotionally shocking than the exaggerations about the ‘creation myth,’ should be much more troubling. CAI and Mortenson’s responses to the accusations so far on 60 Minutes have seemed superficial, and I think it’s safe to say that they will not come out of this looking squeaky clean.
I believe this episode raises two broader questions for the nonprofit community.
First, Krakauer chronicles a string of board members, employees, and consultants who came in, were shocked by how things were done and/or discovered discrepancies, and ended up leaving or resigning in protest. This section (pages 50-51) jumped out at me:
After Mortenson refused to comply with CFO Debbie Raynor’s repeated requests to provide documentation for overseas programs, Raynor contacted Ghulam Parvi (the Pakistan program manager) directly, instructing him to provide her with documentation. For two or three months Parvi complied – until Mortenson found out what was going on and ordered Parvi to stop. Raynor resigned.
In 2007, Mortenson hired an accomplished consultant to periodically fly to Central Asia to supervise projects. When he discovered irregularities and shared them with Mortenson, Mortenson took no action to rectify the misconduct. In 2010, the consultant quit in frustration.
In September 2007, CAI hired a highly motivated, uncommonly capable woman to manage its international programs. Quickly, she demonstrated initiative and other leadership skills the Institute sorely needed. She had exceptional rapport with Pakistani women and girls. In 2008, she unearthed serious issues in Baltistan that contradicted what Mortenson had been reporting. After she told Mortenson about these problems, she assumed he would want her to address them. Instead, as she prepared to return to Pakistan in 2009, Mortenson ordered her to stay away from Baltistan. Disillusioned, she resigned in June 2010.
Seriously — ff this has been going on for so long, how on earth is it just coming out now? Evidently a nationally known organization can have nearly its entire board resign and multiple employees quit, and it doesn’t make the news until years later? Some of this (I’m speculating here) likely results from a hesitance on the part of those former employees to speak ill of CAI, whether because they still believed in its mission or because they were worried about being the sour grape person. Were they speaking out and nobody listened, or is there simply no good way to raise red flags about a nonprofit organization?
Second, while most organizations aren’t guilty of fraud — we hope — there’s at least one other take-away here. Another excerpt that jumped out at me:
On June 13, 2010, Parvi convened a meeting in Skardu to discuss Three Cups of Tea. Some thirty community leaders from throughout Baltistan participated, and most of them were outraged by the excerpts Parvi translated for them. Sheikh Muhammad Raza—chairman of the education committee at a refugee camp in Gultori village, where CAI has built a primary school for girls—angrily proposed charging Mortenson with the crime of fomenting sectarian unrest, and urged the District Administration to ban Mortenson and his books from Baltistan.
Based on Krakauer’s footnotes, Parvi may be one of his less reliable sources, but this idea — that the people portrayed in the book were outraged when it was translated to them because of how misleading it is — comes up several times. Yes, fabricating stories is really bad. But how many other things do nonprofits say in their advertising that would be uncomfortable or downright offensive if you translated it for (and/or showed the accompanying pictures to) the recipients or beneficiaries or their services?
I propose a simple way to check this impulse — to write about people as if they are victims or powerless — and in honor of Three Cups of Tea, I call it the “Tea Test”:
Step One: read the website content, blog posts, or email appeal you just got from your charity of choice. Or, if you work for a nonprofit organization, read your own stuff.
Step Two: imagine arriving in the recipient city or village, with a translated copy of that text. Would you be uncomfortable reading that website or blog or email to the people you met? Would it require tortured explanations, or would it instantly make sense and leave them feeling dignified?
That’s it: if Step Two didn’t make you cringe, then you passed the Tea Test. If it made you uncomfortable, made them feel ashamed, or got you attacked — re-draft your copy and try again. Or find another organization to support.
I think there are many organizations that pass the Tea Test, but probably many more that fail. These organizations don’t necessarily share all the faults of CAI as laid out by Krakauer and others, but they wouldn’t fare much better in this situation, because they say something for one audience that was never intended to get back to the others.
I hope the idea of the Tea Test — reading a translated copy of that material to the people it’s describing — will be helpful for donors and nonprofiteers alike. As a former online fundraiser I know I’ve broken this rule, and as a donor I’ve found things appealing that I probably should have reacted strongly against. I’m going to try to do better.
Update: I’ve posted a slightly revised (and I hope easier to remember) version of the Tea Test on a permanent page here.
William Easterly and Laura Freschi go after “Inception Statistics” in the latest post on AidWatch. They criticize — in typically hyperbolic style, with bonus points for the pun in the title — both the estimates of stillbirth and their coverage in the news media. I left a comment on their blog outlining my thoughts but thought I’d re-post them here with a little more explanation. Here’s what I said:
Thanks for this post (it’s always helpful to look at quality of estimates critically) but I think the direction of your criticism needs to be clarified. Which of the following are you upset about (choose all that apply)?
a) the fact that the researchers used models at all? I don’t know the researchers personally, but I would imagine that they are concerned with data quality in general and would much preferred to have had reliable data from all the countries they work with. But in the absence of that data (and while working towards it) isn’t it helpful to have the best possible estimates on which to set global health policy, while acknowledging their limitations? Based on the available data, is there a better way to estimate these, or do you think we’d be better off without them (in which case stillbirth might be getting even less attention)?
b) a misrepresentation of their data as something other than a model? If so, could you please specify where you think that mistake occurred — to me it seems like they present it in the literature as what it is and nothing more.
c) the coverage of these data in the media? On that I basically agree. It’s helpful to have critical viewpoints on articles where there is legitimate disagreement.
I get the impression your main beef is with (c), in which case I agree that press reports should be more skeptical. But I think calling the data “made up” goes too far too. Yes, it’d be nice to have pristine data for everything, but in the meantime we should try for the best possible estimates because we need something on which to base policy decisions. Along those lines, I think this commentary by Neff Walker (full disclosure: my advisor) in the same issue is worthwhile. Walker asks these five questions – noting areas where the estimates need improvement:
– “Do the estimates include time trends, and are they geographically specific?” (because these allow you to crosscheck numbers for credibility)
– “Are modelled results compared with previous estimates and differences explained?”
– “Is there a logical and causal relation between the predictor and outcome variables in the model?”
– “Do the reported measures of uncertainty around modelled estimates show the amount and quality of available data?”
– “How different are the settings from which the datasets used to develop the model were drawn from those to which the model is applied?” (here Walker says further work is needed)
I’ll admit to being in over my head in evaluating these particular models. As Easterly and Freschi note, “the number of people who actually understand these statistical techniques well enough to judge whether a certain model has produced a good estimate or a bunch of garbage is very, very small.” Very true. But in the absence of better data, we need models on which to base decisions — if not we’re basing our decisions on uninformed guesswork, rather than informed guesswork.
I think the criticism of media coverage is valid. Even if these models are the best ever they should still be reported as good estimates at best. But when Easterly calls the data “made up” I think the hyperbole is counterproductive. There’s an incredibly wide spectrum of data quality, from completely pulled-out-of-the-navel to comprehensive data from a perfectly-functioning vital registration system. We should recognize that the data we work with aren’t perfect. And there probably is a cut-off point at which estimates are based on so many models-within-models that they are hurtful rather than helpful in making informed decisions. But are these particular estimates at that point? I would need to see a much more robust criticism than AidWatch has provided so far to be convinced that these estimates aren’t helpful in setting priorities.
David Simon, creator of The Wire and newly minted MacArthur Fellow, is interviewed by Bill Moyers in Guernica. It’s one of the best things I’ve read in quite a while.
David Simon: You talk honestly with some of the veteran and smarter detectives in Baltimore, the guys who have given their career to the drug war, including, for example, Ed Burns, who was a drug warrior for twenty years, and they’ll tell you, this war’s lost. This is all over but the shouting and the tragedy and the waste. And yet there isn’t a political leader with the stomach to really assess it for what it is.
Bill Moyers: So whose lives are less and less necessary in America today?
David Simon: Certainly the underclass. There’s a reason they are the underclass. We’re in an era when you don’t need as much mass labor; we are not a manufacturing base. People who built stuff, their lives had some meaning and value because the factories were open. You don’t need them anymore.
When I first moved to Baltimore I avoided watching The Wire for several months because I didn’t want it to color my first impressions, and I’ve still only had time to watch the first season. But based on that alone, The Wire was a work of art, and one that was always risky in terms of commercial success because of the length of its story arcs.
A while back Kottke highlighted Simon’s original pitch for the series (emphasis added):
But more than an exercise is realism for its own sake, the verisimilitude of The Wire exists to serve something larger. In the first story-arc, the episodes begin what would seem to be the straight-forward, albeit protracted, pursuit of a violent drug crew that controls a high-rise housing project. But within a brief span of time, the officers who undertake the pursuit are forced to acknowledge truths about their department, their role, the drug war and the city as a whole. In the end, the cost to all sides begins to suggest not so much the dogged police pursuit of the bad guys, but rather a Greek tragedy. At the end of thirteen episodes, the reward for the viewer — who has been lured all this way by a well-constructed police show — is not the simple gratification of hearing handcuffs click. Instead, the conclusion is something that Euripides or O’Neill might recognize: an America, at every level at war with itself.
And not just with itself. The Guernica interview I quoted from above resonated with me because I had just finished this Economist article on “Central America: the tormented isthmus” which outlines the many ways in which America’s appetites and means result in our internal war being continuously foisted upon other countries.
Nearly all the world’s cocaine is produced in Colombia, Peru and Bolivia. The biggest consumer is the United States, where the wholesale price of a kilo of the stuff, even full of impurities, starts at $12,500. The route to market used to run from Colombia to the tip of Florida, across the Caribbean. But the United States Coast Guard shut down that corridor by the early 1990s, and shipments switched to the Pacific coast of Mexico. Now Mexico, too, has increased the pressure on the traffickers, just as Colombia has done in the south.
Ever supple, the drugs business has sought new premises. Somewhere between 250 and 350 tonnes of cocaine—or almost the whole amount heading for the United States—now pass through Guatemala each year, according to American officials…
The impact has been lethal. Guatemala’s murder rate has doubled in the past decade. In both Guatemala and El Salvador, the rate of killing is higher now than during their civil wars.
The comments on that article led me to this BBC article from April 7, on the drug-fueled violence in Mexico.
This view [that the violence is the result of fighting between rival criminal gangs, a sign of progress in the drug war] was echoed by the head of the US Drug Enforcement Administration, Michele Leonhart, at an international conference in the Mexican City of Cancun on Wednesday.
“It may seem contradictory, but the unfortunate level of violence is a sign of success in the fight against drugs,” the DEA chief said.
And that must also mean that the increasing violence in Central America due to shifts in drug trafficking patterns is truly a sign that we’re winning the future. If this is success, maybe we’d all be better of with failure.
My first (and only) full year of coursework for my masters program is drawing to a close. Finals are in mid-May, and comprehensive exams are in early June. Then it’s off to New York City!
On June 8 I will be joining the 5th class of the Epi Scholars program with the New York City Department of Health and Mental Hygiene. Each scholar (read: grad student in epidemiology) is paired with a staff mentor and given a specific project to work on throughout the summer. The research work is augmented by training on SAS and GIS, educational sessions on health disparities and other topics that touch on the work of the Department. The project I will be working on involves characterizing children with severe lead poisoning in New York to help clinicians better screen for them. I’ve only heard amazing things about this program from the Hopkins students and alumni who have gone before, so I’m quite excited.
In the fall I will move to an as-yet-undisclosed location and spend the majority of my second year doing ‘field work’ in global health. One of the reasons I chose this program is that it gives me the chance to get a substantial chunk (6-12 months) of work experience abroad in one place before I complete my degree, and I plan to take full advantage of that. More on where I’ll be once things are finalized.
The blog Stuff Expat Aid Workers Like is really tearing it up lately. Their latest post is#45 – Blogging to Display Their Superior Thinking. An excerpt:
Microlending? Oversold uncritically as a silver bullet and only your Kiva-donating grandma still thinks this is a cure-all. Girl Effect? Undoes its own message with its objectionable messaging. Advocacy? You mean, “badvocacy?” Perilously reductionist and, anyway, spearheaded by way too many celebrities, neo-hippies and naive idealists for it to do any real good. In-kind donations? Logistical nightmare and destroyer of local markets. Popular journalists on the developing country beat (and Nicolas Kristof in particular)? Dangerously oversimplify complex global issues that only the real EAW bloggers truly understand.
The secret and deep hope of the EAW blogger is to get the blessing of the aid blog patriarch, Bill Easterly, and any of his disciples, and get a shout out or, better yet, featured on his blogroll.
Ouch. Of course, none of this applies to me yet because I am not a real expat aid worker, but rather a grad student intending to be one. But hopefully this fall I will be able to start fulfilling this post (which I think is their best yet): #44 – Blogging for the Folks Back Home.
Update/clarification: the “wannabe” in the title is a reference to me. Thought that was clear, but maybe it wasn’t.
Astonishingly, a third of the wealthiest 20% of Indian children are malnourished, too, and they are neither poor nor excluded.
That’s from the Economist last Thursday.
Wonder what it says about India, but also what it says about our measures of wealth and malnutrition.
USAID evidently offers a number of short online courses on global health, including quite a few related to PEPFAR. I just registered but haven’t tried these out yet — if you have, please let me know what you think in the comments. They’re available at www.globalhealthlearning.org.
From an email:
We are pleased to announce the launch of six new eLearning courses on the U.S. Agency for International Development’s Global Health eLearning Center (www.globalhealthlearning.org):
- Healthy Businesses: Familiarizes learners with strategies to design and deliver activities to ensure that commercial for-profit health care providers have the business, operational, and financial capacity to sustainably provide essential health services.
- Male Circumcision: Policy and Programming: Provides learners with an overview of scientific evidence of male circumcision’s (MC’s) protective effect against HIV transmission, the acceptability and safety of MC, challenges to MC program implementation, and policy and program guidance.
PEPFAR-related eLearning courses:
- Data Use for Program Managers: Provides learners with a systematic approach to planning for the use of data, specifically within the field of HIV/AIDS.
- Economic Evaluation Basics: Gives learners a basic understanding of the common methods used to conduct an economic evaluation and the role of economic evaluations in policy and program decision-making in the field of international public health.
- Geographic Approaches to Global Health: Acquaints learners with spatial data and the use of such data to enhance the decision-making process for health program implementation in limited resource settings.
- PEPFAR Next Generation Indicators Guidance: Allows learners to gain a better understanding about the newest version of the NGI Reference Guide and how the information contained in the guide can be used to report progress of PEPFAR programs within national monitoring and evaluation frameworks.
Thank you very much for your interest in and support of the Global Health eLearning Center!
David Brooks highlights a discussion “on what scientific concepts everyone’s cognitive toolbox should hold” on Edge.org. Brooks’ first highlight is this:
Clay Shirkey nominates the Pareto Principle. We have the idea in our heads that most distributions fall along a bell curve (most people are in the middle). But this is not how the world is organized in sphere after sphere. The top 1 percent of the population control 35 percent of the wealth. The top two percent of Twitter users send 60 percent of the messages. The top 20 percent of workers in any company will produce a disproportionate share of the value. Shirkey points out that these distributions are regarded as anomalies. They are not.
The full Edge.org symposium is here. I’m not sure these individual insights are science or even scientific concepts, as much as “insights on thinking that some scientists have found useful” — but still interesting. Here’s Richard Dawkins on the Double-Blind Control Experiment (emphasis added):
….Why do half of all Americans believe in ghosts, three quarters believe in angels, a third believe in astrology, three quarters believe in Hell? Why do a quarter of all Americans and believe that the President of the United States was born outside the country and is therefore ineligible to be President? Why do more than 40 percent of Americans think the universe began after the domestication of the dog?
Let’s not give the defeatist answer and blame it all on stupidity. That’s probably part of the story, but let’s be optimistic and concentrate on something remediable: lack of training in how to think critically, and how to discount personal opinion, prejudice and anecdote, in favour of evidence. I believe that the double-blind control experiment does double duty. It is more than just an excellent research tool. It also has educational, didactic value in teaching people how to think critically. My thesis is that you needn’t actually do double-blind control experiments in order to experience an improvement in your cognitive toolkit. You only need to understand the principle, grasp why it is necessary, and revel in its elegance.
If all schools taught their pupils how to do a double-blind control experiment, our cognitive toolkits would be improved in the following ways:
1. We would learn not to generalise from anecdotes.
2. We would learn how to assess the likelihood that an apparently important effect might have happened by chance alone.
3. We would learn how extremely difficult it is to eliminate subjective bias, and that subjective bias does not imply dishonesty or venality of any kind. This lesson goes deeper. It has the salutary effect of undermining respect for authority, and respect for personal opinion….