Monday miscellany

A day late, but "Tuesday miscellany" loses the alliteration:

  • "Promoting professional networks at work", by Ian Thorpe, who writes the blog "Knowledge Management on a Dollar a Day". This post has a lot of practical advice for organizations, especially when onboarding new staff.
  • "Does it take a village?" by Paul Starobin in Foreign Policy, examines Jeff Sachs, the Millennium Villages Project, and their evaluation plans. For those who have been following the subject for a while there's some interesting background material worked into the piece that I hadn't read before.
  • "On what do health economists agree?" asks the Incidental Economist. The penultimate point (on inequality) is the only one I thought didn't seem widely agreed, and the blog comments concur.
  • Causal inference: extrapolating from sample to population, via the Monkey Cage. This paper argues that findings done with multiple regression from supposedly representative samples aren't necessarily representative; seems likely to become widely read and discussed.
  • Berk Ozler of the World Bank Development Impact blog shares some enlightening comparisons of medical and economic journals.
  • Finally, Hans Rosling explains population growth and climate change (with Legos!).

On economic history

Here’s an excerpt from Peter Temin’s “The Rise and Fall of Economic History at MIT.” (PDF, via Mankiw)

What is the cost of not having economic history at MIT? It can be seen in Acemoglu and Robinson, Why Nations Fail (2012). This is a deservedly successful popular book, making a simple and strong point that the authors made originally at the professional level over a decade before (Acemoglu, Johnson and Robinson, 2001). They assert that countries can be “ruled by a narrow elite that have [sic] organized society for their own benefit at the expense of the vast mass of people” or can have “a revolution that transformed the politics and thus the economics of the nation … to expand their economic opportunities (Acemoglu and Robinson, 2012, pp. 3-4).”

The book is not however good economic history. It is an example of Whig history in which good policies make for progress and bad policies preclude it. Only transitions from bad to good are considered in this colorful but still monotonic story. The clear implication is that if countries can copy the policies of English-speaking countries, they will prosper. No consideration is given to Britain’s economic problems over the past half-century or of Australia’s relative decline for a century.

His take on the US is also rather provocative – worth a read.

This seems like a good time to recommend Evolving Economics, a history of economics that I found helpful for supplementing my crash-course introduction to the field. It’s quite dense (I still haven’t been able to read it straight through) but was a good resource for looking up particular scholars, and for understanding the path-dependency and personalities behind the development of economics.

Americanah

Americanah, the new novel by Chimamanda Ngozi Adichie is very good. I have a long list of Nigerian fiction on my to-read list, but Americanah got bumped to the top because it seemed like the perfect transition from Princeton to Nigeria: I heard Chimamanda speak in Princeton – where she, like Ifemelu, the main character, lived for a year on a fellowship – a month or so ago. Americanah starts with Ifemelu taking NJ Transit from Princeton to Trenton to get her hair braided, because Princeton is the sort of place with an “ice cream shop that had fifty different flavors including red pepper” but no one to braid black hair. Following her TED Talk advice, Americanah crams in many narratives. It’s set in Lagos and London, Brooklyn and Baltimore, New Haven and Philly, and it’s about migration from Lagos to America, from Lagos to London, and from everywhere back to Nigeria. One character, in London:

His eyes would follow them, with a lost longing, and he would think: You can work, you are legal, you are visible, and you don’t even know how fortunate you are.

It’s about dating across race, wealth, and cultures; academics and intellectuals and the many people who are only one or the other, not both; the London black market of arranged sham marriages and faked ID documents; accents real and faked; sex work; the constant burdens and exploitation and desperation of the undocumented; Barack Obama; the hope and opportunity that can come with an approved visa application; and hair. Lots of hair.

There are Americans who deny that racism is still a problem. Wealthy folks who, learning Ifemelu is from Nigeria, try to connect by mentioning their latest trip to Tanzania, their opinion of Ethiopian beauty, the charity they support in Malawi. Ifemelu thinks:

There was a certain luxury to charity that she could not identify with and did not have…. Ifemelu wanted, suddenly and desperately, to be from the country of people who gave and not those who received, to be one of those who had and could therefore bask in the grace of having given, to be among those who could afford copious pithy and empathy.

Another character is at a London dinner party, thinking:

Alexa, and the other guests, and perhaps even Georgina, all understood the fleeing from war, form the kind of poverty that crushed human souls, but they would not understand the need to escape form the oppressive lethargy of choicelessness. They would not understand why people like him, who were raised well fed and watered but mired in dissatisfaction, conditioned from birth to look towards somewhere else, eternally convinced that real lives happened in that somewhere else, were now resolved to do dangerous things, illegal things, so as to leave, none of them starving, or raped, or from burned villages, but merely hungry for choice and certainty.

Ifemelu is, for a while, a blogger who writes “Raceteenth or Various Observations About American Blacks (Those Formerly Known as Negroes) by a Non-American Black” which gives Adichie a venue to make observations, often hilarious and/or impolite. One post starts:

Dear Non-American Black, when you make the choice to come to America, you become black. Stop arguing. Stop saying I’m Jamaican or I’m Ghanaian. America doesn’t care. So what if you weren’t “black” in your country? You’re in America now….

Americanah never dwells on a single theme until it becomes tiresome The major characters are sympathetic but flawed, and the observations are constantly insightful – I wanted to quote much more here. So, highly recommended.

Comparisons

It’s hard for me to experience Nigeria without comparing it – mentally, and probably too often, verbally, with Ethiopia. Or rather, comparing Abuja to Addis, since my experience in each country has been centered on the capital. A few thoughts with a broad brush stroke: compared to Addis, Abuja is hotter (lower altitude), the roads are much better (oil wealth? planned city?), the taxis and most cars are newer (less massive import taxes?), the driving is much more aggressive (cars that can actually go fast + fast roads), the upscale grocery stores have amazing selection (more Nigerian buying power?), and security and crime are much greater, ever-present concerns. The music is better (sorry, Teddy Afro) and the conversation louder. The international scene here is more British, more male, and – especially outside of Abuja – more ensconced in all-encompassing compounds called “life camps” run by big foreign oil and construction companies that, like NGOs, often have 3-letter acronym names that have long outlived their original meaning.

Monday miscellany: on my reading list

NYC and London from the air

My recent New York to London flight featured both good approach paths and nice weather, so I snapped a few shots. Here's New York, taking off:

And London:

I was going to include a quote here from Gotham, an epic and impressively readable history of New York City, regarding how communication and travel times between London and New York decreased during the 17th and 18th centuries, but can't find the quote readily now... So, short version: things got faster.

Next up!

After three years, I'm done with grad school! I finished my MSPH (Global Disease Epidemiology and Control focus) at Hopkins in late May, and my MPA (Economics and Public Policy focus) at Princeton in early June. It's been a lot of work: 10 months of internships, 3 comprehensive qualifying exams, and a Masters thesis; plus 4 quarters of Hopkins classwork and 3 semesters of Princeton classwork for a total of 33 graduate classes. I loved being in school again -- not all my classmates did -- but I'm also happy to have wrapped things up. One consequence of studying applied subjects like public health and public policy is that you're rarely delving into a subject just for kicks (at least for long); the goal is always to get out and do good work with the knowledge and skills you've acquired. This week I started a job I'm really excited about: working with the Clinton Health Access Initiative (CHAI)'s Applied Analytics Team. If you're curious about CHAI here's their about page, and this profile of Elizabeth McCarthy tells a bit more about the Applied Analytics Team (which she runs). We're also hiring. As with my previous internships and work, I won't be writing directly about what I'm doing much at all, but I'll still be writing more broadly about global health and development policy. (And this is probably a good time to reiterate that the views here are just my own.) I'll be working on projects throughout sub-Saharan Africa -- I'm headed to Nigeria for a couple months on Saturday! More on that soon.

Now, back to my (ir)regular blogging...

Several job opps

Some other good places to look for jobs: mHealth student Google group and the African Development Jobs blog.

"What is wrong (and right) in economics?"

Economist Dani Rodrik has a great essay up on his website on what's good and bad about economics. Here's a bit on the relationship between trade policy and growth:

I remember well the reception I got when I presented my paper (with Francisco Rodriguez) on the empirics of trade policy and growth. The literature had filled up with extravagant claims about the effect of trade liberalization on economic growth. What we showed in our paper is that the research to date could not support those claims. Neither the theoretical nor empirical literature indicated there is a robust, predictable, and quantitatively large effect of trade liberalization on growth. We were simply stating what any well-trained economist should have known. Nevertheless, the paper was highly controversial. One of my Harvard colleagues asked me in the Q&A session: “why are you doing this?” It was a stunning question. It was as if knowledge of a certain kind was dangerous.

There's a lot of good material in there about what economics is and isn't, and how to do it better.  I had forgotten that Rodrik studied at Princeton, so was pleasantly surprised by this:

However, contemporary economics in North America has one great weakness, and that is the excessive focus on methods at the expense of breadth in terms of social and historical perspective. PhD programs now train applied mathematicians and statisticians rather than real economists. To become a true economist, you need to do all sorts of reading – from history, sociology, and political science among other disciplines – that you are never required to do as a graduate student. The best economists today find a way of filling this gap in their education. I consider myself very lucky that I was a political science major and did a master’s in public affairs (as it is called at Princeton) before I turned to economics. I say lucky, because some of my best work – by my judgement, at least – was stimulated by questions or arguments I encountered outside of neoclassical economics.

"When public health works, it's invisible"

Caitlin Rivers' post on the "public health paradox: why people don't get flu shots" hits the nail on the head:

Unfortunately, the root of this problem is deep. The problem is that when public health works, it is invisible. It's an insidious, persistent public relations issue that plagues public health. Nobody sees when a chain of disease transmission is broken, or when contaminated food is prevented from reaching the market, or when toxic pollutants don't enter the environment. That's the point: the goal of public health is prevention, not reaction....

What then can be done to counteract these misperceptions? First, public health needs to be more vocal about its successes. This graphic of crude death rates for infectious diseases during the 19th century, for example, should be widely disseminated. A little self-promotion could go a long ways.

That's one reason I like Millions Saved, from the Center for Global Development -- it highlights "proven success in global health." One of the things that struck me when reading it was that most of the people who benefited from these interventions and programs would have no way of knowing that they benefited.

For another positive take, check out Charles Kenny's book Getting Better.

 

Monday miscellany

(Not) knowing it all along

David McKenzie is one of the guys behind the World Bank's excellent and incredibly wonky Development Impact blog. He came to Princeton to present on a new paper with Gustavo Henrique de Andrade and Miriam Bruhn, "A Helping Hand or the Long Arm of the Law? Experimental evidence on what governments can do to formalize firms" (PDF). The subject matter -- trying to get small, informal companies to register with the government -- is outside my area of expertise. But I thought there were a couple methodologically interesting bits: First, there's an interesting ethical dimension, as one of their several interventions tested was increasing the likelihood that a firm would be visited by a government inspector (i.e., that the law would be enforced). From page 10:

In particular, if a firm owner were interviewed about their formality status, it may not be considered ethical to then use this information to potentially assign an inspector to visit them. Even if it were considered ethical (since the government has a right to ask firm owners about their formality status, and also a right to conduct inspections), we were still concerned that individuals who were interviewed in a baseline survey and then received an inspection may be unwilling to respond to a follow-up. Therefore a listing stage was done which did not involve talking to the firm owner.

In other words, all their baseline data was collected without actually talking to the firms they were studying -- check out the paper for more on how they did that.

Second, they did something that could (and maybe should) be incorporated into many evaluations with relative ease. Because findings often seem obvious after we hear them, McKenzie et al. asked the government staff whose program they were evaluating to estimate what the impact would be before the results were in. Here's that section (emphasis added):

A standard question with impact evaluations is whether they deliver new knowledge or merely formally confirm the beliefs that policymakers already have (Groh et al, 2012). In order to measure whether the results differ from what was anticipated, in January 2012 (before any results were known) we elicited the expectations of the Descomplicar [government policy] team as to what they thought the impacts of the different treatments would be. Their team expected that 4 percent of the control group would register for SIMPLES [the formalization program] between the baseline and follow-up surveys. We see from Table 7 that this is an overestimate...

They then expected the communication only group to double this rate, so that 8 percent would register, that the free cost treatment would lead to 15 percent registering, and that the inspector treatment would lead to 25 percent registering.... The zero or negative impacts of the communication and free cost treatments therefore are a surprise. The overall impact of the inspector treatment is much lower than expected, but is in line with the IV estimates, suggesting the Descomplicar team have a reasonable sense of what to expect when an inspection actually occurs, but may have overestimated the amount of new inspections that would take place. Their expectation of a lack of impact for the indirect inspector treatment was also accurate.

This establishes exactly what in the results was a surprise and what wasn't. It might also make sense for researchers to ask both the policymakers they're working with and some group of researchers who study the same subject to give such responses; it would certainly help make a case for the value of (some) studies.

An uphill battle

I took this photo in the NYC subway a few days ago. My apologies for the quality, but I thought it's a great juxtaposition:

In the top of the photo is an ad from the NYC Department of Health, advising you to choose food with less sodium. (Here's an AP story about the ads.) But to the bottom right is an ad for McDonald's dollar menu, and those are everywhere. While it doesn't mean we shouldn't run such ads, it's worth remembering that the sheer volume of food advertising will always dwarf opposing health messages. 

Fun projects are fun

Jay Ulfelder, of the blog Dart-Throwing Chimp, recently wrote a short piece in praise of fun projects. He links to my Hunger Games survival analysis, and Alex Hanna's recent application of survival analysis to a reality TV show, RuPaul's Drag Race. (That single Hunger Games post has accounted for about one-third of the ~100k page views this blog got in the last year!) Jay's post reminded me that I never shared links to Alex's survival analysis, which is a shame, so here goes: First, there's "Lipsyncing for your life: a survival analysis of RuPaul's Drag Race":

I don’t know if this occurs with other reality shows (this is the first I’ve been taken with), but there is some element of prediction involved in knowing who will come out as the winner. A drag queen we spoke with at Plan B suggested that the length of time each queen appears in the season preview is an indicator, while Homoviper’s “index” is largely based on a more qualitative, hermeneutic analysis. I figured, hey, we could probably build a statistical model to know which factors are the most determinative in winning the competition.

And then come two follow-ups, where Alex digs into predictions for the next episode of the current season, and again for the one after that. That last post is a great little lesson on the importance of the proportional hazards assumption.

I strongly agree with this bit from Jay's post about the value of these projects:

Based on personal experience, I’m a big believer in learning by doing. Concepts don’t stick in my brain when I only read about them; I’ve got to see the concepts in action and attach them to familiar contexts and examples to really see what’s going on.

Right on. And in addition to being useful, these projects are, well, fun!

Rearranging the malarial deck chairs?

A friend sent this link to me, highlighting a critical comment about the future of the World Health Organization, in the context of the World Malaria Report 2012. Here's an excerpt of the comment by William Jobin:

Their 2012 Annual Report is a very disturbing report from WHO, for at least two reasons:

1. Their program is gradually falling apart, and they offer no way to refocus, no strategy for dealing with the loss in funding, nor the brick wall of drug and biocide resistance which is just down the road. There is a label for people who keep doing the same thing, but expect different results. Do you remember what it is?

2. Because the entire top management of WHO consists of physicians, they have no idea of the opportunities they are missing for additional funding and for additional methods to add to their chemically-oriented strategy...

Concluding with:

I am not sure WHO has much of a future, nor does the UN system itself, after their failure to prevent the wars in Libya and Syria. But as long as the UN and WHO continue to operate, they must refocus their approach to face the reality of a rapidly declining budget from UN sources. Instead, I see them just re-arranging the deck chairs on the Titanic.

My friend said, "I wish these comments (and issues with the WHO and UN) were more publicised! This is not the first time I am hearing of such issues with the WHO and its demise." I've certainly heard similar sentiments about the WHO from classmates and professors, but it seems there's much less open discussion than you might expect. I'd welcome discussion in the comments...

Smartphones on the cheap

Here's a quick digression from global health that I thought might be interesting to to tech-minded folks. nsnippets, a fascinating link blog (found via Blattman) has a post called "China's 65 dollar smartphones" that caught my attention, because I (sort of) have one of these phones. That post is highlighting a  Technology Review piece: "Here's where they make China's cheap Android smartphones." And here's more on even cheaper phones.

Before moving to Ethiopia I was stuck in a Tmobile contract that was poor value for money with a glitchy phone. Since I'm only back in the US for about 5 months finishing my last semester of grad school I resolved to get an unlocked phone that I could use in the US or abroad, on whatever network I liked, and at a grad student price. I bought one on Amazon from "China Global Inc." and shipped by some third party directly from China. The exact model isn't available anymore but you can find similar phones by searching on Amazon for "Unlocked Quad Band Dual Sim Android 4.0 OS." It gets some incredible double-take reactions because it looks almost exactly like an iPhone in front, but on the back it has the Android logo and just says "Smartphone":

It cost just $135, and I use a $30/month prepaid plan (also Tmobile) with 100 minutes of talk (which is about right for my usage), unlimited text, and unlimited data -- and I'm not locked in at all. My annual cost for this Android smartphone: $495. If you buy an iPhone 5 on Verizon your annual costs are, depending on your contract, in the $920 to $1400 range! I'm sure for some the differences between what I have and a brand new iPhone 5 with 4G (my phone is 3G) are worth $500-1000 annually, but it works for texting, email, search, Twitter, music, games, and so forth -- everything I want.

I can't imagine that everyone with the latest smartphone actually 'needs it' -- in the sense that if they knew there were good alternatives they would think the difference is worth the value. American phone plans are generally incredibly overpriced, leaving you stuck in a cycle of buying premium products -- which are nice -- but ironically being locked into keeping them until they're well past premium.  I think what is happening is that as long as most of your friends have high-priced phones with expensive contracts, that's the norm and the price seems less absurd.

This beautiful graphic is not really that useful

This beautiful infographic from the excellent blog Information is Beautiful has been making the rounds. You can see a bigger version here, and it's worth poking around for a bit. The creators take all deaths from the 20th century (drawing from several sources) and represent their relative contribution with circles:

I appreciate their footnote that says the graphic has "some inevitable double-counting, broad estimation and ball-park figures." That's certainly true, but the inevitably approximate nature of these numbers isn't my beef.

The problem is that I don't think raw numbers of deaths tell us very much, and can actually be quite misleading. Someone who saw only this infographic might well end up less well-informed than if they didn't see it. Looking at the red circles you get the impression that non-communicable and infectious diseases were roughly equivalent in importance in the 20th century, followed by "humanity" (war, murder, etc) and cancer.

The root problem is that mortality is inevitable for everyone, everywhere. This graphic lumps together pneumonia deaths at age 1 with car accidents at age 20, and cancer deaths at 50 with heart disease deaths at 80. We typically don't  (and I would argue should't) assign the same weight to a death in childhood or the prime of life with one that comes at the end of a long, satisfying life.  The end result is that this graphic greatly overemphasizes the importance of non-communicable diseases in the 20th century -- that's the impression most laypeople will walk away with.

A more useful graphic might use the same circles to show the years of life lost (or something like DALYs or QALYs) because those get a bit closer at what we care about. No single number is actually  all that great, so we can get a better understanding if we look at several different outcomes (which is one problem with any visualization). But I think raw mortality numbers are particularly misleading.

To be fair, this graphic was commissioned by Wellcome as "artwork" for a London exhibition, so maybe it should be judged by a different standard...

First responses to DEVTA roll in

In my last post I highlighted the findings from the DEVTA trial of deworming in Vitamin A in India, noting that the Vitamin A results would be more controversial. I said I expected commentaries over the coming months, but we didn't have to wait that long after all. First is a BBC Health Check program features a discussion of DEVTA with Richard Peto, one of the study's authors. It's for a general audience so it doesn't get very technical, and because of that it really grated when they described this as a "clinical trial," as that has certain connotations of rigor that aren't reflected in the design of the study. If DEVTA is a clinical trial, then so was

Peto also says there were two reasons for the massive delay in publishing the trial, 1) time to check things and "get it straight," and 2) that they were " afraid of putting up a trial with a false negative." [An aside for those interested in publication bias issues: can you imagine an author with strong positive findings ever saying the same thing about avoiding false positives?!]

Peto ends by sounding fairly neutral re: Vitamin A (portraying himself in a middle position between advocates in favor and skeptics opposed) but acknowledges that with their meta-analysis results Vitamin A is still "cost-effective by many criteria."

Second is a commentary in The Lancet by Al Sommers, Keith West, and Reynaldo Martorell. A little history: Sommers ran the first big Vitamin A trials in Sumtra (published in 1986) and is the former dean of the Johns Hopkins School of Public Health.  (Sommers' long-term friendship with Michael Bloomberg, who went to Hopkins as an undergrad, is also one reason the latter is so big on public health.) For more background, here's a recent JHU story on Sommers' receiving a $1 million research prize in part for his work on Vitamin A.

Part of their commentary is excerpted below, with my highlights in bold:

But this was neither a rigorously conducted nor acceptably executed efficacy trial: children were not enumerated, consented, formally enrolled, or carefully followed up for vital events, which is the reason there is no CONSORT diagram. Coverage was ascertained from logbooks of overworked government community workers (anganwadi workers), and verified by a small number of supervisors who periodically visited randomly selected anganwadi workers to question and examine children who these workers gathered for them. Both anganwadi worker self-reports, and the validation procedures, are fraught with potential bias that would inflate the actual coverage.

To achieve 96% coverage in Uttar Pradesh in children found in the anganwadi workers' registries would have been an astonishing feat; covering 72% of children not found in the anganwadi workers' registries seems even more improbable. In 2005—06, shortly after DEVTA ended, only 6·1% of children aged 6—59 months in Uttar Pradesh were reported to have received a vitamin A supplement in the previous 6 months according to results from the National Family Health Survey, a national household survey representative at national and state level.... Thus, it is hard to understand how DEVTA ramped up coverage to extremely high levels (and if it did, why so little of this effort was sustained). DEVTA provided the anganwadi workers with less than half a day's training and minimal if any incentive.

They also note that the study funding was minimalist compared to more rigorous studies, which may be an indication of quality. And as an indication that there will almost certainly be alternative meta-analyses that weight the different studies differently:

We are also concerned that Awasthi and colleagues included the results from this study, which is really a programme evaluation, in a meta-analysis in which all of the positive studies were rigorously designed and conducted efficacy trials and thus represented a much higher level of evidence. Compounding the problem, Awasthi and colleagues used a fixed-effects analytical model, which dramatically overweights the results of their negative findings from a single population setting. The size of a study says nothing about the quality of its data or the generalisability of its findings.

I'm sure there will be more commentaries to follow. In my previous post I noted that I'm still trying to wrap my head around the findings, and I think that's still right. If I had time I'd dig into this a bit more, especially the relationship with the Indian National Family Health Survey. But for now I think it's safe to say that two parsimonious explanations for how to reconcile DEVTA with the prior research are emerging:

1. DEVTA wasn't all that rigorous and thus never achieved the high population coverage levels necessary to have a strong mortality impact; the mortality impact was attenuated by poor coverage, resulting in the lack of a statistically significant effect in line with prior results. Thus is shouldn't move our priors all that much. (Sommers et al. seem to be arguing for this.) Or,

2. There's some underlying change in the populations between the older studies and these newer studies that causes the effect of Vitamin A to decline -- this could be nutrition, vaccination status, shifting causes of mortality, etc. If you believe this, then you might discount studies because they're older.

(h/t to @karengrepin for the Lancet commentary.)

A massive trial, a huge publication delay, and enormous questions

It's been called the "largest clinical* trial ever": DEVTA (Deworming and Enhanced ViTamin A supplementation), a study of Vitamin A supplementation and deworming in over 2 million children in India, just published its results. "DEVTA" may mean "deity" or "divine being" in Hindi but some global health experts and advocates will probably think these results come straight from the devil. Why? Because they call into question -- or at least attenuate -- our estimates of the effectiveness of some of the easiest, best "bang for the buck" interventions out there. Data collection was completed in 2006, but the results were just published in The Lancet. Why the massive delay? According to the accompany discussion paper, it sounds like the delay was rooted in very strong resistance to the results after preliminary outcomes were presented at a conference in 2007. If it weren't for the repeated and very public shaming by the authors of recent Cochrane Collaboration reviews, we might not have the results even today. (Bravo again, Cochrane.)

So, about DEVTA. In short, this was a randomized 2x2 factorial trial, like so:

The results were published as two separate papers, one on Vitamin A and one on deworming, with an additional commentary piece:

The controversy is going to be more about what this trial didn't find, rather than what they did: the confidence interval on the Vitamin A study's mortality estimate (mortality ratio 0.96, 95% confidence interval of 0.89 to 1.03) is consistent with a mortality reduction as large as 11%, or as much as a 3% increase. The consensus from previous Vitamin A studies was mortality reductions of 20-30%, so this is a big surprise. Here's the abstract to that paper:

Background

In north India, vitamin A deficiency (retinol <0·70 μmol/L) is common in pre-school children and 2–3% die at ages 1·0–6·0 years. We aimed to assess whether periodic vitamin A supplementation could reduce this mortality.

Methods

Participants in this cluster-randomised trial were pre-school children in the defined catchment areas of 8338 state-staffed village child-care centres (under-5 population 1 million) in 72 administrative blocks. Groups of four neighbouring blocks (clusters) were cluster-randomly allocated in Oxford, UK, between 6-monthly vitamin A (retinol capsule of 200 000 IU retinyl acetate in oil, to be cut and dripped into the child’s mouth every 6 months), albendazole (400 mg tablet every 6 months), both, or neither (open control). Analyses of retinol effects are by block (36 vs36 clusters).

The study spanned 5 calendar years, with 11 6-monthly mass-treatment days for all children then aged 6–72 months.  Annually, one centre per block was randomly selected and visited by a study team 1–5 months after any trial vitamin A to sample blood (for retinol assay, technically reliable only after mid-study), examine eyes, and interview caregivers. Separately, all 8338 centres were visited every 6 months to monitor pre-school deaths (100 000 visits, 25 000 deaths at ages 1·0–6·0 years [the primary outcome]). This trial is registered at ClinicalTrials.gov, NCT00222547.

Findings

Estimated compliance with 6-monthly retinol supplements was 86%. Among 2581 versus 2584 children surveyed during the second half of the study, mean plasma retinol was one-sixth higher (0·72 [SE 0·01] vs 0·62 [0·01] μmol/L, increase 0·10 [SE 0·01] μmol/L) and the prevalence of severe deficiency was halved (retinol <0·35 μmol/L 6% vs13%, decrease 7% [SE 1%]), as was that of Bitot’s spots (1·4% vs3·5%, decrease 2·1% [SE 0·7%]).

Comparing the 36 retinol-allocated versus 36 control blocks in analyses of the primary outcome, deaths per child-care centre at ages 1·0–6·0 years during the 5-year study were 3·01 retinol versus 3·15 control (absolute reduction 0·14 [SE 0·11], mortality ratio 0·96, 95% CI 0·89–1·03, p=0·22), suggesting absolute risks of death between ages 1·0 and 6·0 years of approximately 2·5% retinol versus 2·6% control. No specific cause of death was significantly affected.

Interpretation

DEVTA contradicts the expectation from other trials that vitamin A supplementation would reduce child mortality by 20–30%, but cannot rule out some more modest effect. Meta-analysis of DEVTA plus eight previous randomised trials of supplementation (in various different populations) yielded a weighted average mortality reduction of 11% (95% CI 5–16, p=0·00015), reliably contradicting the hypothesis of no effect.

Note that instead of just publishing these no-effect results and leaving the meta-analysis to a separate publication, the authors go ahead and do their own meta-analysis of DEVTA plus previous studies and report that -- much attenuated, but still positive -- effect in their conclusion. I think that's a fair approach, but also reveals that the study's authors very much believe there are large Vitamin A mortality effects despite the outcome of their own study!

[The only media coverage I've seen of these results so far comes from the Times of India, which includes quotes from the authors and Abhijit Banerjee.]

To be honest, I don't know what to make of the inconsistency between these findings and previous studies, and am writing this post in part to see what discussion it generates. I imagine there will be more commentaries on these findings over the coming months, with some decrying the results and methodologies and others seeing vindication in them. In my view the best possible outcome is an ongoing concern for issues of external validity in biomedical trials.

What do I mean? Epidemiologists tend to think that external validity is less of an issue in randomized trials of biomedical interventions -- as opposed to behavioral, social, or organizational trials -- but this isn't necessarily the case. Trials of vaccine efficacy have shown quite different efficacy for the same vaccine (see BCG and rotavirus) in different locations, possibly due to differing underlying nutritional status or disease burdens. Our ability to interpret discrepant findings can only be as sophisticated as the available data allows, or as sophisticated as allowed by our understanding of the biological and epidemiologic mechanisms that matter on the pathway from intervention to outcome. We can't go back in time and collect additional information (think nutrition, immune response, baseline mortality, and so forth) on studies far in the past, but we can keep such issues in mind when designing trials moving forward.

All that to say, these results are confusing, and I look forward to seeing the global health community sort through them. Also, while the outcomes here (health outcomes) are different from those in the Kremer deworming study (education outcomes), I've argued before that lack of effect or small effects on the health side should certainly influence our judgment of the potential education outcomes of deworming.

*I think given the design it's not that helpful to call this a 'clinical' trial at all - but that's another story.