First responses to DEVTA roll in

In my last post I highlighted the findings from the DEVTA trial of deworming in Vitamin A in India, noting that the Vitamin A results would be more controversial. I said I expected commentaries over the coming months, but we didn’t have to wait that long after all.

First is a BBC Health Check program features a discussion of DEVTA with Richard Peto, one of the study’s authors. It’s for a general audience so it doesn’t get very technical, and because of that it really grated when they described this as a “clinical trial,” as that has certain connotations of rigor that aren’t reflected in the design of the study. If DEVTA is a clinical trial, then so was

Peto also says there were two reasons for the massive delay in publishing the trial, 1) time to check things and “get it straight,” and 2) that they were ” afraid of putting up a trial with a false negative.” [An aside for those interested in publication bias issues: can you imagine an author with strong positive findings ever saying the same thing about avoiding false positives?!]

Peto ends by sounding fairly neutral re: Vitamin A (portraying himself in a middle position between advocates in favor and skeptics opposed) but acknowledges that with their meta-analysis results Vitamin A is still “cost-effective by many criteria.”

Second is a commentary in The Lancet by Al Sommers, Keith West, and Reynaldo Martorell. A little history: Sommers ran the first big Vitamin A trials in Sumtra (published in 1986) and is the former dean of the Johns Hopkins School of Public Health.  (Sommers’ long-term friendship with Michael Bloomberg, who went to Hopkins as an undergrad, is also one reason the latter is so big on public health.) For more background, here’s a recent JHU story on Sommers’ receiving a $1 million research prize in part for his work on Vitamin A.

Part of their commentary is excerpted below, with my highlights in bold:

But this was neither a rigorously conducted nor acceptably executed efficacy trial: children were not enumerated, consented, formally enrolled, or carefully followed up for vital events, which is the reason there is no CONSORT diagram. Coverage was ascertained from logbooks of overworked government community workers (anganwadi workers), and verified by a small number of supervisors who periodically visited randomly selected anganwadi workers to question and examine children who these workers gathered for them. Both anganwadi worker self-reports, and the validation procedures, are fraught with potential bias that would inflate the actual coverage.

To achieve 96% coverage in Uttar Pradesh in children found in the anganwadi workers’ registries would have been an astonishing feat; covering 72% of children not found in the anganwadi workers’ registries seems even more improbable. In 2005—06, shortly after DEVTA ended, only 6·1% of children aged 6—59 months in Uttar Pradesh were reported to have received a vitamin A supplement in the previous 6 months according to results from the National Family Health Survey, a national household survey representative at national and state level…. Thus, it is hard to understand how DEVTA ramped up coverage to extremely high levels (and if it did, why so little of this effort was sustained). DEVTA provided the anganwadi workers with less than half a day’s training and minimal if any incentive.

They also note that the study funding was minimalist compared to more rigorous studies, which may be an indication of quality. And as an indication that there will almost certainly be alternative meta-analyses that weight the different studies differently:

We are also concerned that Awasthi and colleagues included the results from this study, which is really a programme evaluation, in a meta-analysis in which all of the positive studies were rigorously designed and conducted efficacy trials and thus represented a much higher level of evidence. Compounding the problem, Awasthi and colleagues used a fixed-effects analytical model, which dramatically overweights the results of their negative findings from a single population setting. The size of a study says nothing about the quality of its data or the generalisability of its findings.

I’m sure there will be more commentaries to follow. In my previous post I noted that I’m still trying to wrap my head around the findings, and I think that’s still right. If I had time I’d dig into this a bit more, especially the relationship with the Indian National Family Health Survey. But for now I think it’s safe to say that two parsimonious explanations for how to reconcile DEVTA with the prior research are emerging:

1. DEVTA wasn’t all that rigorous and thus never achieved the high population coverage levels necessary to have a strong mortality impact; the mortality impact was attenuated by poor coverage, resulting in the lack of a statistically significant effect in line with prior results. Thus is shouldn’t move our priors all that much. (Sommers et al. seem to be arguing for this.) Or,

2. There’s some underlying change in the populations between the older studies and these newer studies that causes the effect of Vitamin A to decline — this could be nutrition, vaccination status, shifting causes of mortality, etc. If you believe this, then you might discount studies because they’re older.

(h/t to @karengrepin for the Lancet commentary.)

1 Comments Add Yours ↓

The upper is the most recent comment

  1. Prof Umesh Kapil #
    1

    Title: The Impact of Vitamin A Supplementation on Under 5 Mortality in Programmatic Conditions

    The research manuscript published online in Lancet entitled “Vitamin A supplementation every 6 months with retinol in 1 million pre-school children in north India: DEVTA, a cluster-randomised trial” by Shally Awasthi et al.(1) has further added to the scientific evidence on the impact of Vitamin A Supplementation ( VAS) on Under 5 Mortality.

    The children in the study area were given VAS at six monthly interval . This intervention resulted in the increase in the serum retinol levels and reduction in the prevalence of the Bitots spot in the experimental group . These two findings confirms that the VAS was undertaken meticulously amongst children in the areas selected for intervention

    The study concluded that the VAS under programmatic conditions had no impact on the under five mortality. The comments made by Sommer etal , in correspondence section of the Lancet, raised issues on the research methodology of the study. These may be valid , however, this does not negate the findings of the research study.

    The impact of VAS has to occur in the programmatic conditions to have benefits in the populations . The efficacy trial are adequate for generating evidence but the ultimate aim is to replicate the efficacy trials and have impact in the programmatic conditions.

    In India, earlier efficacy study, conducted by National Institute of Nutrition , Hyderabad(2) also did not revealed any impact of VAS on Under five mortality.

    Each country has it’s own epidemiological variables , what is true for Indonesia may not be true for India. A specific intervention may work in Africa and may not work in India ., The Study by Awsthi et al proves this hypotheses.

    Umesh Kapil

    AS Bhadoria

    References

    Awasthi S, Peto R, Read S, Clark S, Pande V, Bundy D, and the DEVTA
    (Deworming and Enhanced Vitamin A) team. Vitamin A supplementation every 6 months with retinol in 1 million pre-school children in north India: DEVTA, a cluster-randomised trial. Lancet 2013; published online March 14. http://dx.doi.org/10.1016/S0140-6736(12)62125-4.

    2 Vijayaraghavan K, Radhaiah G, Prakasam BS, Sarma KVR, Reddy V. Eff ect of massive dose vitamin A on morbidity and mortality in Indian children. Lancet 1990; 336: 1342–45.



Your Comment to Prof Umesh Kapil