Data: big, small, and meta

When I read this New York Times piece back in August, I was in the midst of preparation and training for data collection at rural health facilities in Zambia. The Times piece profiles a group called Global Pulse that is doing good work on the 'big data' side of global health:

The efforts by Global Pulse and a growing collection of scientists at universities, companies and nonprofit groups have been given the label “Big Data for development.” It is a field of great opportunity and challenge. The goal, the scientists involved agree, is to bring real-time monitoring and prediction to development and aid programs. Projects and policies, they say, can move faster, adapt to changing circumstances and be more effective, helping to lift more communities out of poverty and even save lives.

Since I was gearing up for 'field work' (more on that here; I'll get to it soon), I was struck at the time by the very different challenges one faces at the other end of the spectrum. Call it small data? And I connected the Global Pulse profile with this, by Wayan Vota, from just a few days before:

The Sneakernet Reality of Big Data in Africa

When I hear people talking about “big data” in the developing world, I always picture the school administrator I met in Tanzania and the reality of sneakernet data transmissions processes.

The school level administrator has more data than he knows what to do with. Years and years of student grades recorded in notebooks – the hand-written on paper kind of notebooks. Each teacher records her student attendance and grades in one notebook, which the principal then records in his notebook. At the local district level, each principal’s notebook is recorded into a master dataset for that area, which is then aggregated at the regional, state, and national level in even more hand-written journals... Finally, it reaches the Minister of Education as a printed-out computer-generated report, complied by ministerial staff from those journals that finally make it to the ministry, and are not destroyed by water, rot, insects, or just plain misplacement or loss. Note that no where along the way is this data digitized and even at the ministerial level, the data isn’t necessarily deeply analyzed or shared widely....

And to be realistic, until countries invest in this basic, unsexy, and often ignored level of infrastructure, we’ll never have “big data” nor Open Data in Tanzania or anywhere else. (Read the rest here.)

Right on. And sure enough two weeks later I found myself elbow-deep in data that looked like this -- "Sneakernet" in action:

In many countries a quite a lot of data -- of varying quality -- exists, but it's often formatted like the above. Optimistically, it may get used for local decisions, and eventually for high-level policy decisions when it's months or years out of date. There's a lot of hard, good work being done to improve these systems (more often by residents of low-income countries, sometimes by foreigners), but still far too little. This data is certainly primary, in the sense that was collected on individuals, or by facilities, or about communities, but there are huge problems with quality, and with the sneakernet by which it gets back to policymakers, researchers, and (sometimes) citizens.

For the sake of quick reference, I keep a folder on my computer that has -- for each of the countries I work in -- most of the major recent ultimate sources of nationally-representative health data. All too often the only high-quality ultimate source is the most recent Demographic and Health Survey, surely one of the greatest public goods provided by the US government's aid agency. (I think I'm paraphrasing Angus Deaton here, but can't recall the source.) When I spent a summer doing epidemiology research with the New York City Department of Health and Mental Hygiene, I was struck by just how many rich data sources there were to draw on, at least compared to low-income countries. Very often there just isn't much primary data on which to build.

On the other end of the spectrum is what you might call the metadata of global health. When I think about the work the folks I know in global health -- classmates, professors, acquaintances, and occasionally thought not often me -- do day to day, much of it is generating metadata. This is research or analysis derived from the primary data, and thus relying on its quality. It's usually smart, almost always well-intentioned, and often well-packaged, but this towering edifice of effort is erected over a foundation of primary data; the metadata sometimes gives the appearance of being primary, when you dig down the sources often point back to those one or three ultimate data sources.

That's not to say that generating this metadata is bad: for instance, modeling impacts of policy decisions given the best available data is still the best way to sift through competing health policy priorities if you want to have the greatest impact. Or a more cynical take: the technocratic nature of global health decision-making requires that we either have this data or, in its absence, impute it. But regardless of the value of certain targeted bits of the metadata, there's the question of the overall balance of investment in primary vs. secondary-to-meta data, and my view -- somewhat ironically derived entirely from anecdotes -- is that we should be investing a lot more in the former.

One way to frame this trade-off is to ask, when considering a research project or academic institute or whatnot, whether the money spent on that project might result in more value for money if it was spent instead training data collectors and statistics offices, or supporting primary data collection (e.g., funding household surveys) in low-income countries. I think in many cases the answer will be clear, perhaps to everyone except those directly generating the metadata.

That does not mean that none of this metadata is worthwhile. On the contrary, some of it is absolutely essential. But a lot isn't, and there are opportunity costs to any investment, a choice between investing in data collection and statistics systems in low-income countries, vs. research projects where most of the money will ultimately stay in high-income countries, and the causal pathway to impact is much less direct.  

Looping back to the original link, one way to think of the 'big data' efforts like Global Pulse is that they're not metadata at all, but an attempt to find new sources of primary data. Because there are so few good sources of data that get funded, or that filter through the sneakernet, the hope is that mobile phone usage and search terms and whatnot can be mined to give us entirely new primary data, on which to build new pyramids of metadata, and with which to make policy decisions, skipping the sneakernet altogether. That would be pretty cool if it works out.

Monday miscellany

  • If you're in DC this Wednesday, Charles Kenny is giving a talk to launch his new book, The Upside of Down: Why the Rise of the Rest Is Good for the WestCharles' previous book (Getting Better) was a good read and he's been churning out interesting journalism since (sample here). You might describe the general theme of Charles' writing as "not everything is all that terrible," which is remarkable because so much of the writing -- at least the smart writing -- on international development could be summarized as "really, everything is quite terrible."
  • Newspapers are biased (study) towards covering medical articles that aren't as good. (Via the always interesting Justin Wolfers.) Another way of reading this is that the higher quality papers are typically RCTs, but many of the questions that are most interesting to the lay public can only be answered with large observational studies. Those studies are more likely to give answers that won't hold up to further study, and more likely to be dreadfully overhyped by their authors and by journalists.
  • Angus Deaton reviews Nina Munk's book on Jeff Sachs. Sachs is not impressed. Two thoughts: 1) I love that Deaton connects it to the Anti-Politics Machine, which is one of the best books on development and what I kept thinking of on reading The Idealist. 2) A three-way conversation between Sachs, Deaton, and Michael Clemens would be fascinating, in part because Deaton and Clemens are both Sachs critics, but differ strongly on RCTs -- Clemens has written about how the Millennium Villages could be evaluated with them, and Deaton wouldn't be impressed even if they were.
  • Elizabeth Pisani (author of the Wisdom of Whores) has a new book soon on Indonesia.
  • Some humor: What if meetings were all like conference calls, and The Onion describes the new American Dream.
  • Bill Gates shared this graph on Twitter, showing how the distribution of log GDP per capita has changed from a bimodal "camel" distribution to a single dome today. (It might be even more informative to look at the same numbers with and without China, which accounts for much of the departure from absolute poverty):

A more useful aid debate

Ken Opalo highlights recent entries on the great aid debate from Bill Gates, Jeff Sachs, Bill Easterly, and Chris Blattman. Much has been said on this debate, and sometimes it feels like it's hard to add anything new. But since having a monosyllabic first name seem sufficient qualification to weigh in, I will. First, this part of Ken's post resonates with me:

I think most reasonable people would agree that Sachs kind of oversold his big push idea in The End of Poverty. Or may be this was just a result of his attempt to shock the donor world into reaching the 0.7 percent mark in contributions. In any event it is unfortunate that the debate on the relative efficacy of aid left the pages of journal articles in its current form. It would have been more helpful if the debate spilled into the public in a policy-relevant form, with questions like: under what conditions does aid make a difference? What can we do to increase the efficacy of aid? What kinds of aid should we continue and what kinds should we abolish all together? (emphasis added)

Lee Crawfurd wrote something along these lines too: "Does Policy Work?"  Lee wrote that on Jan 10, 2013, and I jokingly said it was the best aid blog post of the year (so far). Now that 2013 has wrapped up, I'll extend that evaluation to 'best aid blog post of 2013'. It's worth sharing again:

The question "does policy work" is jarring, because we immediately realise that it makes little sense. Governments have about 20-30 different Ministries, which immediately implies at least 20-30 different areas of policy. Does which one work? We have health and education policy, infrastructure policy (roads, water, energy), trade policy, monetary policy, public financial management, employment policy, disaster response, financial sector policy, climate and environment policy, to name just a few. It makes very little sense to ask if they all collectively "work" or are "effective". Foreign aid is similar. Aid supports all of these different areas of policy....

A common concern is about the impact of aid on growth... Some aid is specifically targeted at growth - such as financing infrastructure or private sector development. But much of it is not. One of the few papers which looks at the macroeconomic impact of aid and actually bothers to disaggregate even a little the different types of aid, finds that the aid that could be considered to have growth as a target, does increase growth. It's the aid that was never intended to impact growth at all, such as humanitarian assistance, which doesn't have any impact on growth.

I like to think that most smart folks working on these issues -- and that includes both Sachs and Easterly -- would agree with the following summaries of our collective state of knowledge:

  •  A lot of aid projects don't work, and some of them do harm.
  • Some aid, especially certain types of health projects, works extremely well.

The disagreement is on the balance of good and bad, so I wish -- as Ken wrote -- the debate spilled into the public sphere along those lines (which is good? which is bad? how can we get a better mix?) rather than the blanket statements both sides are driven to by the very publicness of the debate. It reminds me a bit of debates in theology: if you put a fundamentalist and Einstein in the same room, they'll both be talking about "God" but meaning very different things with the same words. (This is not a direct analogy, so don't ask who is who...)

When Sachs and Easterly talk about whether aid "works", it would be nice if we could get everyone to first agree on a definition of "aid" and "works". But much of this seems to be driven by personal animosity between Easterly and Sachs, or more broadly, by personal animosity of a lot of aid experts vs. Sachs. Why's that? I think part of the answer is that it's hard to tell when Sachs is trying to be a scientist, and when he's trying to be an advocate. He benefits from being perceived as the former, but in reality is much more the latter. Nina Munk's The Idealist -- an excellent profile of Sachs I've been meaning to review -- explores this tension at some length. The more scientifically-minded get riled up by this confusion -- rightfully, I think. At the same time, public health folks tend to love Sachs precisely because he's been a powerful advocate for some types of health aid that demonstrably work -- also rightfully, I think. There's a tension there, and it's hard to completely dismiss one side as wrong, because the world is complicated and there are many overlapping debates and conversations; academic and lay, public and private, science and advocacy.

So, back to Ken's questions that would be answered by a more useful aid debate:

  • Under what conditions does aid make a difference?
  • What can we do to increase the efficacy of aid?
  • What kinds of aid should we continue and what kinds should we abolish all together?

Wouldn't it be amazing if the public debate were focused on these questions? Actually, something like that was done: Boston Review had a forum a while back on "Making Aid Work" with responses by Abhijit Banerjee, Angus Deaton, Howard White, Ruth Levine, and others. I think that series of questions is much more informative than another un-moderated round of Sachs vs Easterly.

Year in review - infographic style!

It's been about six months since I wrote a real blog post other than a link round-up. One of my 2014 resolutions is to write more regularly -- either for this blog or for myself -- and I'm calling on you, blog readers, to hold me to it. In the meantime, I wanted to share a bit about what it was that kept me too busy to blog. It was a jam-packed year between finishing school, starting a new job, and traveling for fun and for work. At some point in the fall I made a pie chart of where I had spent time so far in the year, and that led to the idea of doing a holiday greeting card in the form of an infographic. I put one together over the holidays and share it with friends and family -- it's supposed to be a bit over the top and tongue in cheek, and it might just become an annual tradition, though future versions will have much better metrics. Click for the PDF:

 

I also updated the Photography page with links to these albums from 2013: EthiopiaCosta RicaCape Town, and Lesotho. I'm sure there's a better way to present some of these, so suggestions in the comments for integrating photography into a blog are welcome.

Monday miscellany

I'm hoping to start writing posts other than link round-ups soon, but I've been swamped the last few month with fascinating work and travels in the US, Nigeria, Zambia, South Africa, and Lesotho. Not to mention the latest Hunger Games movie is giving me lots of ideas for a follow-up to my original survival analysis!  More on that soon. In the meantime:

 

 

Monday miscellany: just give people money edition

Unconditional cash transfers (i.e., just giving people money) are the talk of the blogosphere right now:

The post that comes closest to my own feelings is from Matt Collin of Aid Thoughts: "When every argument begins with 'is it better than cash?'" Here's an excerpt:

...there are a range of public goods (or semi-private goods which have substantial externalities) which we can imagine might increase welfare a great deal more than a cash transfer of equivalent cost: schools, health facilities, roads, a functioning police force. Basically, any semblance of a local or national state. How many of you would vote for your own government to transfer its entire budget evenly across the population and then shut down all its operation for good? It certainly would make it easier to pay the rent next month, if your apartment complex hadn’t been burned down by the marauding hordes yet.

A bandwagon

It's great, read the whole thing. I wouldn't go as far as Matt in saying that there is a "current bias towards cash transfers", but otherwise I agree. There's growing evidence that cash transfers, conditional or not, work pretty well. That means they're probably better than many forms of institutional aid, some of which don't have evidence of working at all. That doesn't mean they're better than all forms of institutional assistance, or that all forms are even directly comparable to cash. Many public services in health -- I'm thinking especially of vaccination and other investments in preventative health -- are unlikely to materialize in response to cash transfers alone. In short, I think it's useful for both individual and institutional donors to think in terms of portfolios: considering giving cash directly, but also simultaneously investing in the public provision of services.

Some older, related posts:

Monday miscellany

Spreading the word

If you haven't already read Atul Gawande's latest New Yorker piece on why some ideas spread fast and other spread slow, get to it:

 In the era of the iPhone, Facebook, and Twitter, we’ve become enamored of ideas that spread as effortlessly as ether. We want frictionless, “turnkey” solutions to the major difficulties of the world—hunger, disease, poverty. We prefer instructional videos to teachers, drones to troops, incentives to institutions. People and institutions can feel messy and anachronistic. They introduce, as the engineers put it, uncontrolled variability.

But technology and incentive programs are not enough. “Diffusion is essentially a social process through which people talking to people spread an innovation,” wrote Everett Rogers, the great scholar of how new ideas are communicated and spread. Mass media can introduce a new idea to people. But, Rogers showed, people follow the lead of other people they know and trust when they decide whether to take it up. Every change requires effort, and the decision to make that effort is a social process.

Much of the material is Gawande's essay won't be new if you're already interested in or working on maternal and child health, but Gawande presents it incredibly well. His comparison of spreading social innovation with the work of salesman also reminded me of another parallel: the parallels between diffusing secular, health-enhancing ideas and missionaries' evangelistic techniques.

If that last sentence scares you off, hold on a moment for some background. I grew up in a small religious town in Arkansas and my first trips to developing countries were as a missionary. Over time my interests shifted from the preaching and teaching side of things to the medical side, and eventually to health and development policy as an entirely secular pursuit. When I first got to grad school for public health this resulted in some awkward moments, as many conversations would start with "so what first interested you in global health?" If I led with "well, I grew up wanting to be a missionary" I would often get one of two reactions: immediate skepticism of my motivations from my secular liberal classmates, or enthusiastic endorsement of my work (as they misunderstood it) from religious classmates. All that to say: while I think there are very good general reasons to keep public health and missionary efforts as separate as possible, both in theory and praxis, there are several things we secular liberals can still learn from the more devout.

One example is the neverending debates amongst evangelists between those who seek technological shortcuts and those who stick with old-fashioned person-to-person contact. This is a frequent topic at missions conferences (if you didn't know such conferences existed, it might be an interesting google). You can view the rise of Christian radio broadcasts, followed by Christian TV and televangelists, as the great technological shortcuts: they give a single preacher the ability to reach millions, and if the message is just as good as when delivered in person, why shouldn't it be just as effective? Some people are persuaded by televangelists, of course, but the effectiveness of the individual doesn't scale easily to mass media. Likewise, in recent years there's been much enthusiasm for social media and its potential to save more souls -- but the results rarely pan out.  So despite all of the advances in mass and social media, most evangelists still harp on the importance of individual contact, of building relationships. One of the most effective (in terms of growth rate) groups in the world are Mormons, who, no coincidence, devote years of effort to one-on-one contact.

Gawande's essay tells the story of how BRAC precipitated oral rehydration solution in Bangladesh, and I couldn't help thinking of their campaign  as a sort of especially successful roving gospel meeting. And here's Gawande's closing, where he talks with a nurse who was convinced by a younger, less-experienced trainer to adopt some best practices for safe childbirth:

 “Why did you listen to her?” I [Gawande] asked. “She had only a fraction of your experience.”

In the beginning, she didn’t, the nurse admitted. “The first day she came, I felt the workload on my head was increasing.” From the second time, however, the nurse began feeling better about the visits. She even began looking forward to them.

“Why?” I asked.

All the nurse could think to say was “She was nice.”

“She was nice?”

“She smiled a lot.”

“That was it?”

“It wasn’t like talking to someone who was trying to find mistakes,” she said. “It was like talking to a friend.”

Shortcuts are nice: in public health, unlike evangelism, it's usually actions rather than beliefs that ultimately count, so I'm all for technological shortcuts when they're available and effective. But they're too few and far between, and much of the low-hanging fruit in global health has already been picked. To climb the next step require a lot more effort at improving the "messy and anachronistic"  processes of people and institutions.

Formalizing corruption: US medical system edition

Oh, corruption. It interferes with so many aspects of daily life, adding time to the simplest daily tasks, costing more money, and -- often the most frustrating aspect -- adding huge doses of uncertainty. That describes life in many low-income, high-corruption countries, leading to many a conversation with friends about comparisons with the United States and other wealthy countries. How did the US "solve" corruption? I've heard (and personally made) the argument that the US reduced corruption at least in part by formalizing it; by channeling the root of corruption, a sort of rent-seeking on a personal level, to rent-seeking on an institutional level. The US political and economic system has evolved such that some share of any wealth created is channeled into the pockets of a political and economic elite who benefit from the system and in turn reinforce it. That unproductively-channeled share of wealth is simultaneously a) probably smaller than the share of wealth lost to corruption in most developing countries, b) still large enough to head off -- along with the threat of more effective prosecution -- at least some more overt corruption, and c) still a major drain on society.

An example: Elisabeth Rosenthal profiles medical tourism in an impressive series in the New York Times. In part three of the series, an American named Michael Shopenn travels to Belgium to get a hip replacement. Why would he need to? Because health economics in the US is less a story of free markets and  more a story of political capture by medical interests, including technology and pharmaceutical companies, physicians' groups, and hospitals:

Generic or foreign-made joint implants have been kept out of the United States by trade policy, patents and an expensive Food and Drug Administration approval process that deters start-ups from entering the market. The “companies defend this turf ferociously,” said Dr. Peter M. Cram, a physician at the University of Iowa medical school who studies the costs of health care.

Though the five companies make similar models, each cultivates intense brand loyalty through financial ties to surgeons and the use of a different tool kit and operating system for the installation of its products; orthopedists typically stay with the system they learned on. The thousands of hospitals and clinics that purchase implants try to bargain for deep discounts from manufacturers, but they have limited leverage since each buys a relatively small quantity from any one company.

In addition, device makers typically require doctors’ groups and hospitals to sign nondisclosure agreements about prices, which means institutions do not know what their competitors are paying. This secrecy erodes bargaining power and has allowed a small industry of profit-taking middlemen to flourish: joint implant purchasing consultants, implant billing companies, joint brokers. There are as many as 13 layers of vendors between the physician and the patient for a hip replacement, according to Kate Willhite, a former executive director of the Manitowoc Surgery Center in Wisconsin.

If this system existed in another country we wouldn't hesitate to call it corrupt, and to note that it actively hurts consumers. It should be broken up by legislation for the public good, but instead it's protected by legislators who are lobbied by the industry and by doctors who receive kickbacks, implicit and explicit. Contrast that with the Belgian system:

His joint implant and surgery in Belgium were priced according to a different logic. Like many other countries, Belgium oversees major medical purchases, approving dozens of different types of implants from a selection of manufacturers, and determining the allowed wholesale price for each of them, for example. That price, which is published, currently averages about $3,000, depending on the model, and can be marked up by about $180 per implant. (The Belgian hospital paid about $4,000 for Mr. Shopenn’s high-end Zimmer implant at a time when American hospitals were paying an average of over $8,000 for the same model.)

“The manufacturers do not have the right to sell an implant at a higher rate,” said Philip Boussauw, director of human resources and administration at St. Rembert’s, the hospital where Mr. Shopenn had his surgery. Nonetheless, he said, there was “a lot of competition” among American joint manufacturers to work with Belgian hospitals. “I’m sure they are making money,” he added.

It's become a cliche to compare the US medical system to European ones, but those comparisons are made because it's hard to realize just how systematically corrupt -- and expensive, as a result -- the US system is without comparing it to ones that do a better job of channeling the natural profit-seeking goals of individuals and companies towards the public good. (For the history of how we got here, Paul Starr is a good place to start.)

The usual counterargument for protecting such large profit margins in the US is that they drive innovation, which is true but only to an extent. And for the implants industry that argument is much less compelling since many of the newer, "innovative" products have proved somewhere between no better and much worse in objective tests.

The Times piece is definitely worth a read. While I generally prefer the formalized corruption to the unformalized version, I'll probably share this article with friends -- in Nigeria, or Ethiopia, or wherever else the subject comes up next.

Advocates and scientists

A new book by The Idealist: Jeffrey Sachs and the Quest to End Poverty. The blurbs on Amazon are fascinating because they indicate that either the reviewers didn't actually read the book (which wouldn't be all that surprising) or that Munk's book paints a nuanced enough picture that readers can come away with very different views on what it actually proves. Here are two examples:

Amartya Sen: “Nina Munk’s book is an excellent – and moving – tribute to the vision and commitment of Jeffrey Sachs, as well as an enlightening account of how much can be achieved by reasoned determination.”

Robert Calderisi: "A powerful exposé of hubris run amok, drawing on touching accounts of real-life heroes fighting poverty on the front line."

The publisher's description seems to encompass both of those points of view: "The Idealist is the profound and moving story of what happens when the abstract theories of a brilliant, driven man meet the reality of human life." That sounds like a good read to me -- I look forward to reading when it comes out in September.

Munk's previous reporting strikes a similar tone. For example, here's an excerpt of her 2007 Vanity Fair profile of Sachs:

Leaving the region of Dertu, sitting in the back of an ancient Land Rover, I'm reminded of a meeting I had with Simon Bland, head of Britain's Department for International Development in Kenya. Referring to the Millennium Villages Project, and to Sachs in particular, Bland laid it out for me in plain terms: "I want to say, 'What concept are you trying to prove?' Because I know that if you spend enough money on each person in a village you will change their lives. If you put in enough resources—enough foreigners, technical assistance, and money—lives change. We know that. I've been doing it for years. I've lived and worked on and managed [development] projects.

"The problem is," he added, "when you walk away, what happens?"

Someone -- I think it was Chris Blattman, but I can't find the specific post -- wondered a while back whether too much attention has been given to the Millennium Villages Project. After all, the line of thinking goes, the MVP's have really just gotten more press and aren't that different from the many other projects with even less rigorous evaluation designs. That's certainly true: when journalists and aid bloggers debate the MVPs, part of what they're debating is Sachs himself because he's such a polarizing personality. If you really care about aid policy, and the uses of evidence in that policy, then that can all feel like an unhelpful distraction. Most aid efforts don't get book-length profiles, and the interest in Sachs' personality and persona will probably drive the interest in Munk's book.

But I also think the MVP debates have been healthy and interesting -- and ultimately deserving of most of the heat generated -- because they're about a central tension within aid and development, as well as other fields where research intersects with activism. If you think we already generally know what to do, then it makes sense to push forward with it at all costs. The naysayers who doubt you are unhelpful skeptics who are on some level ethically culpable for blocking good work. If you think the evidence is not yet in, then it makes more sense to function more like a scientist, collecting the evidence needed to make good decisions in the longer term. The naysayers opposing the scientists are then utopian advocates who throw millions at unproven projects. I've seen a similar tension within the field of public health, between those who see themselves primarily as advocates and those who see themselves as scientists, and I'm sure it exists elsewhere as well.

That is, of course, a caricature -- few people fall completely on one side of the advocates vs. scientists divide. But I think the caricature is a useful one for framing arguments. The fundamental disagreement is usually not about whether evidence should be used to inform efforts to end poverty or improve health or advance any other goal. Instead, the disagreement is often over what the current state of knowledge is. And on that note, if you harbor any doubts on where Sachs has positioned himself on that spectrum here's the beginning of Munk's 2007 profile:

In the respected opinion of Jeffrey David Sachs.... the problem of extreme poverty can be solved. In fact, the problem can be solved "easily." "We have enough on the planet to make sure, easily, that people aren't dying of their poverty. That's the basic truth," he tells me firmly, without a doubt.

...To Sachs, the end of poverty justifies the means. By hook or by crook, relentlessly, he has done more than anyone else to move the issue of global poverty into the mainstream—to force the developed world to consider his utopian thesis: with enough focus, enough determination, and, especially, enough money, extreme poverty can finally be eradicated.

Once, when I asked what kept him going at this frenzied pace, he snapped back, "If you haven't noticed, people are dying. It's an emergency."

----

via Gabriel Demombynes.

If you're new to the Millennium Villages debate, here's some background reading: a recent piece in Foreign Policy by Paul Starobin, and some good posts by Chris Blattman (one, two, three), this gem from Owen Barder, and Michael Clemens.

CHAI jobs

While I write this blog in a personal capacity, I thought I'd point out a few open positions at the Clinton Health Access Initiative and especially on CHAI's Applied Analytics Team, which I joined in early June:

And there are two open positions with the Applied Analytics Team:

  • Senior Research Associate (Masters or PhD) working on Demand-Driven Evaluation for Decisions (3DE), and
  • Health Economist. Ideally this position would be filled by someone with a PhD in economics, modeling skills, and HIV experience in public health, policy, or very applied research – the sort of person who can think outside of their academic specialty and wants to work on applied research questions that policy makers needed answers to yesterday.

There are many more positions posted on the careers section of the CHAI site, though unfortunately there's no RSS feed of posted positions.

Monday miscellany

Uninformative paper titles: "in Africa"

When I saw a new NBER working paper titled "Disease control, demographic change and institutional development in Africa" (PDF) pop up in the NBER RSS feed I thought the title sounded interesting, so I downloaded the paper to peruse later. Then today the new-ish (and great!) blog Cherokee Gothic highlighted the same paper in a post, and I finally took a look. Unfortunately the paper title is rather uninformative, as the authors only used data from Burkina Faso. Sure, economics papers tend to have bigger, less formal titles than papers in some other fields, but I think this is particularly unhelpful. There are enough search frictions in finding applicable literature on any given topic that it helps to be somewhat more precise.

For reference, here's Burkina Faso:

And here's Africa:

Not the same.

It's unclear from the data and arguments presented how these results -- for a regional disease control program, but only using data from Burkina Faso -- might generalize to the quite diverse disease environments, demographic trends, and institutional histories of various African countries. The paper doesn't answer or even give much grounds for speculation on whether onchocerciasis or other disease control programs would yield similar results in countries as diverse as (for example) Senegal, Ethiopia, Uganda, and Angola.

A quick thought experiment: Virginia's population is about 1.5% of the total population of North America, just as Burkina Faso's population is about 1.5% of the total population on Africa. Can you imagine someone writing a paper on health and institutions using data from Virginia and titling that paper "Health and institutions in North America"? Or writing a paper on Vietnamese history and titling it "A history of Asia"? Probably not.

The Napoleon cohort

I've recently had to think through two problems related to tracking cohorts over time, and each time I've mentally referred back to what is considered by some to be the greatest data visualization of all time. Charles Joseph Minard, an engineer, created the graphic below: "Carte figurative des pertes successives en hommes de l'Armée Française dans la campagne de Russie 1812-1813" (loosely translated as "don't follow Napoleon or anyone else when launching a land war in Asia").

This single picture shows the size of the army as it entered Russia, then the size as it left, their relative geographic location, groups leaving and re-entering the force, and the temperature the army faced as they returned.  And to me it meets one of the main tests for "is this graphic great?" -- it sticks in my head and I find myself referring back to it again and again.

"Redefining global health delivery"

Jim Yong Kim, Paul Farmer, and Michael Porter wrote a piece called "Redefining global health delivery" for the Lancet in May. The abstract:

Initiatives to address the unmet needs of those facing both poverty and serious illness have expanded significantly over the past decade. But many of them are designed in an ad-hoc manner to address one health problem among many; they are too rarely assessed; best practices spread slowly. When assessments of delivery do occur, they are often narrow studies of the cost-effectiveness of a single intervention rather than the complex set of them required todeliver value to patients and their families. We propose a framework for global health-care delivery and evaluation by considering efforts to introduce HIV/AIDS care to resource-poor settings. The framework introduces the notion of care delivery value chains that apply a systems-level analysis to the complex processes and interventions that must occur, across a health-care system and over time, to deliver high-value care for patients with HIV/AIDS and cooccurring conditions, from tuberculosis to malnutrition. To deliver value, vertical or stand-alone projects must be integrated into shared delivery infrastructure so that personnel and facilities are used wisely and economies of scale reaped. Two other integrative processes are necessary for delivering and assessing value in global health: one is the alignment of delivery with local context by incorporating knowledge of both barriers to good outcomes (from poor nutrition to a lack of water and sanitation) and broader social and economic determinants of health and wellbeing (jobs, housing, physical infrastructure). The second is the use of effective investments in care delivery to promote equitable economic development, especially for those struggling against poverty and high burdens of disease. We close by reporting our own shared experience of seeking to move towards a science of delivery by harnessing research and training to understand and improve care delivery.

I think the overall thrust of the piece is something that is widely agreed upon by global health policy wonks, but I like that they lay out a more specific framework for thinking with this sort of systems approach. But, I'd love to see some more detail on putting it into practice on a national or subnational level.

Slow down there

Max Fisher has a piece in the Washington Post presenting "The amazing, surprising, Africa-driven demographic future of the Earth, in 9 charts". While he notes that the numbers are "just projections and could change significantly under unforeseen circumstances" the graphs don't give any sense of the huge uncertainty involved in projecting trends out 90 years in the future. Here's the first graph:

 

The population growth in Africa here is a result of much higher fertility rates, and a projected slower decline in those rates.

But those projected rates have huge margins of error. Here's the total fertility rate, or "the average number of children that would be born to a woman over her lifetime"  for Nigeria, with confidence intervals that give you a sense of just how little we know about the future:

That's a lot of uncertainty! (Image from here, which I found thanks to a commenter on the WaPo piece.)

It's also worth noting that if you had made similar projections 87 years ago, in 1926, it would have been hard to anticipate World War II, hormonal birth control, and AIDS, amongst other things.

Typhoid counterfactuals

An acquaintance (who doesn't work in public health) recently got typhoid while traveling. She noted that she had had the typhoid vaccine less than a year ago but got sick anyway. Surprisingly to me, even though she knew "the vaccine was only about 50% effective" she now felt that it was  a mistake to have gotten the vaccine. Why? "If you're going to get the vaccine and still get typhoid, what's the point?" I disagreed but am afraid my defense wasn't particularly eloquent in the moment: I tried to say that, well, if it's 50% effective and you and, I both got the vaccine, then only one of us would get typhoid instead of both of us. That's better, right? You just drew the short straw. Or, if you would have otherwise gotten typhoid twice, now you'll only get it once!

These answers weren't reassuring in part because thinking counterfactually -- what I was trying to do -- isn't always easy. Epidemiologists do this because they're typically told ad nauseum to approach causal questions by first thinking "how could I observe the counterfactual?" At one point after finishing my epidemiology coursework I started writing a post called "The Top 10 Things You'll Learn in Public Health Grad School" and three or four of the ten were going to be "think counterfactually!"

A particularly artificial and clean way of observing this difference -- between what happened and what could have otherwise happened -- is to randomly assign people to two groups (say, vaccine and placebo). If the groups are big enough to average out any differences between them, then the differences in sickness you observe are due to the vaccine. It's more complicated in practice, but that's where we get numbers like the efficacy of the typhoid vaccine -- which is actually a bit higher than 50%.

You can probably see where this is going: while the randomized trial gives you the average effect, for any given individual in the trial they might or might not get sick. Then, because any individual is assigned only to the treatment or control, it's hard to pin their outcome (sick vs. not sick) on that alone. It's often impossible to get an exhaustive picture of individual risk factors and exposures so as to explain exactly which individuals will get sick or not in advance. All you get is an average, and while the average effect is really, really important, it's not everything.

This is related somewhat to Andrew Gelman's recent distinction between forward and reverse causal questions, which he defines as follows:

1. Forward causal inference. What might happen if we do X? What are the effects of smoking on health, the effects of schooling on knowledge, the effect of campaigns on election outcomes, and so forth?

2. Reverse causal inference. What causes Y? Why do more attractive people earn more money? Why do many poor people vote for Republicans and rich people vote for Democrats? Why did the economy collapse?

The randomized trial tries to give us an estimate of the forward causal question. But for someone who already got sick, the reverse causal question is primary, and the answer that "you were 50% less likely to have gotten sick" is hard to internalize. As Gelman says:

But reverse causal questions are important too. They’re a natural way to think (consider the importance of the word “Why”) and are arguably more important than forward questions. In many ways, it is the reverse causal questions that lead to the experiments and observational studies that we use to answer the forward questions.

The moral of the story -- other than not sharing your disease history with a causal inference buff -- is that reconciling the quantitative, average answers we get from the forward questions with the individual experience won't always be intuitive.

African population density

I was recently struck by differences in population density: Northern Nigeria's Kano state has an official population of ~10 million, whereas the entire country of Zambia has 13.5. Zambia's land area, meanwhile, is also about 35 times that of Kano. So I started looking around for a nice map of population density in Africa. The best I found was this one via UNEP:

And here's a higher resolution version.

Some of the most striking concentrations are along the Mediterranean coast, the Nile basin, the Ethiopian plateau, and around Lake Victoria. (I'd love to track down the data behind this map but haven't had time.)

A good map can change how you think. If you're used to seeing maps that have country-level estimates of disease prevalence, for instance, you miss variations at the subnational level. This is often for good reason, as the subnational data is often even spottier than the national estimates. But another thing you miss is a sense of absolute population numbers, because looking at a map it's much easier to see countries by their areas rather than their populations, which for matters of health and other measures of human well-being is generally what we care about. There are some cool maps that do this but they inevitably lose their geographic accuracy.

Transatlantic

Gumbo, a favorite food in the southern United States, particularly in the Louisiana area, is a variation of popular west African (including Yoruba) stews in which similar ingredients, such as okra and spicy peppers, are served over a starchy substance. In west African stews the starch is usually yam or cassava; in gumbo it is usually rice. West African language patterns have also merged with the English language over time. For example, the Yoruba language does not conjugate verbs. Therefore, the English “I am,” you are,” he/she/it is,” translates into Yoruba simply as “emi ni,” iwo ni,” and “oun ni” respectively. Scholars equate this lack of conjugation with colloquial African-American speech patterns that would conjugate the same phrase in English as “I be,” “you be,” he/she/it be,” representing the retention of African language patterns over time and space….

From A History of Nigeria by Falola and Heaton.