Archive for the ‘public policy’Category

When randomization is strategic

Here’s a quote from Tom Yates on his blog Sick Populations about a speech he heard by Rachel Glennerster of J-PAL:

Glennerster pointed out that the evaluation of PROGRESA, a conditional cash transfer programme in Mexico and perhaps the most famous example of randomised evaluation in social policy, was instigated by a Government who knew they were going to lose the next election. It was a way to safeguard their programme. They knew the next Government would find it hard to stop the trial once it was started and were confident the evaluation would show benefit, again making it hard for the next Government to drop the programme. Randomisation can be politically advantageous.

I think I read this about Progresa / Oportunidades before but had forgotten it, and thus it’s worth re-sharing. The way in which Progresa was randomized (different areas were stepped into the program, so there was a cohort of folks who got it later than others, but all the high need areas got it within a few years) made this more politically feasible as well. I think this situation, in which a government institutes a study of a program to keep it alive through subsequent changes of government, will probably be a less common tactic than its opposite, in which a government designs an evaluation of a popular program that a) it thinks doesn’t work, b) it wants to cut, and c) the public otherwise likes, just to prove that it should be cut — but only time will tell.

16

08 2012

Aid, paternalism, and skepticism

Bill Easterly, the ex-blogger who just can’t stop, writes about a conversation he had with GiveWell, a charity reviewer/giving guide that relies heavily on rigorous evidence to pick programs to invest in. I’ve been meaning to write about GiveWell’s approach — which I generally think is excellent. Easterly, of course, is an aid skeptic in general and a critic of planned, technocratic solutions in particular. Here’s an excerpt from his notes on his conversation with GiveWell:

…a lot of things that people think will benefit poor people (such as improved cookstoves to reduce indoor smoke, deworming drugs, bed nets and water purification tablets) {are things} that poor people are unwilling to buy for even a few pennies. The philanthropy community’s answer to this is “we have to give them away for free because otherwise the take-up rates will drop.” The philosophy behind this is that poor people are irrational. That could be the right answer, but I think that we should do more research on the topic. Another explanation is that the people do know what they’re doing and that they rationally do not want what aid givers are offering. This is a message that people in the aid world are not getting.

Later, in the full transcript, he adds this:

We should try harder to figure out why people don’t buy health goods, instead of jumping to the conclusion that they are irrational.

Also:

It’s easy to catch people doing irrational things. But it’s remarkable how fast and unconsciously people get things right, solving really complex problems at lightning speed.

I’m with Easterly, up to a point: aid and development institutions need much better feedback loops, but are unlikely to develop them for reasons rooted in their nature and funding. The examples of bad aid he cites are often horrendous. But I think this critique is limited, especially on health, where the RCTs and all other sorts of evidence really do show that we can have massive impact — reducing suffering and death on an epic scale — with known interventions. [Also, a caution: the notes above are just notes and may have been worded differently if they were a polished, final product — but I think they’re still revealing.]

Elsewhere Easterly has been more positive about the likelihood of benefits from health aid/programs in particular, so I find it quite curious that his examples above of things that poor people don’t always price rationally are all health-related. Instead, in the excerpts above he falls back on that great foundational argument of economists: if people are rational, why have all this top-down institutional interference? Well, I couldn’t help contrasting that argument with this quote highlighted by another economist, Tyler Cowen, at Marginal Revolution:

Just half of those given a prescription to prevent heart disease actually adhere to refilling their medications, researchers find in the Journal of American Medicine. That lack of compliance, they estimate, results in 113,00 deaths annually.

Let that sink in for a moment. Residents of a wealthy country, the United States, do something very, very stupid. All of the RCTs show that taking these medicines will make them live longer, but people fail to overcome the barriers at hand to take something that is proven to make them live longer. As a consequence they die by the hundreds of thousands every single year. Humans may make remarkably fast unconscious decisions correctly in some spheres, sure, but it’s hard to look at this result and see any way in which it makes much sense.

Now think about inserting Easterly’s argument against paternalism (he doesn’t specifically call it that here, but has done so elsewhere) in philanthropy here: if people in the US really want to live, why don’t they take these medicines? Who are we to say they’re irrational? That’s one answer, but maybe we don’t understand their preferences and should avoid top-down solutions until we have more research.

reductio ad absurdum? Maybe. On the one hand, we do need more research on many things, including medication up-take in high- and low-income countries. On the other hand, aid skepticism that goes far enough to be against proven health interventions just because people don’t always value those interventions rationally seems to line up a good deal with the sort of anti-paternalism-above-all streak in conservatism that opposes government intervention in pretty much every area. Maybe it’s a good policy to try out some nudge-y (libertarian paternalism, if you will) policies to encourage people to take their medicine, or require people to have health insurance they would not choose to buy on their own.

Do you want to live longer? I bet you do, and it’s safe to assume that people in low-income countries do as well. Do you always do exactly what will help you do so? Of course not: observe the obesity pandemic. Do poor people really want to suffer from worms or have their children die from diarrhea? Again, of course not. While poor people in low-income countries aren’t always willing to invest a lot of time or pay a lot of money for things that would clearly help them stay alive for longer, that shouldn’t be surprising to us. Why? Because the exact same thing is true of rich people in wealthy countries.

People everywhere — rich and poor — make dumb decisions all the time, often because those decisions are easier in the moment due to our many irrational cognitive and behavioral tics. Those seemingly dumb decisions usually reveal the non-optimal decision-making environments in which we live, but you still think we could overcome those things to choose interventions that are very clearly beneficial. But we don’t always. The result is that sometimes people in low-income countries might not pay out of pocket for deworming medicine or bednets, and sometimes people in high-income countries don’t take their medicine — these are different sides of the same coin.

Now, to a more general discussion of aid skepticism: I agree with Easterly (in the same post) that aid skeptics are a “feature of the system” that ultimately make it more robust. But it’s an iterative process that is often frustrating in the moment for those who are implementing or advocating for specific programs (in my case, health) because we see the skeptics as going too far. I’m probably one of the more skeptical implementers out there — I think the majority of aid programs probably do more harm than good, and chose to work in health in part because I think that is less true in this sector than in others. I like to think that I apply just the right dose of skepticism to aid skepticism itself, wringing out a bit of cynicism to leave the practical core.

I also think that there are clear wins, supported by the evidence, especially in health, and thus that Easterly goes too far here. Why does he? Because his aid skepticism isn’t simply pragmatic, but also rooted in an ideological opposition to all top-down programs. That’s a nice way to put it, one that I think he might even agree with. But ultimately that leads to a place where you end up lumping things together that are not the same, and I’ll argue that that does some harm. Here are two examples of aid, both more or less from Easterly’s post:

  • Giving away medicines or bednets free, because otherwise people don’t choose to invest in them; and,
  • A World Bank project in Uganda that “ended up burning down farmers’ homes and crops and driving the farmers off the land.”

These are a both, in one sense, paternalistic, top-down programs, because they are based on the assumption that sometimes people don’t choose to do what is best for themselves. But are they the same otherwise? I’d argue no. One might argue that they come from the same place, and an institution that funds the first will inevitably mess up and do the latter — but I don’t buy that strong form of aid skepticism. And being able to lump the apparently good program and the obviously bad together is what makes Easterly’s rhetorical stance powerful.

If you so desire, you could label these two approaches as weak coercion and strong coercion. They are both coercive in the sense that they reshape the situations in which people live to help achieve an outcome that someone — a planner, if you will — has decided is better. All philanthropy and much public policy is coercive in this sense, and those who are ideologically opposed to it have a hard time seeing the difference. But to many of us, it’s really only the latter, obvious harm that we dislike, whereas free medicines don’t seem all that bad. I think that’s why aid skeptics like Easterly group these two together, because they know we’ll be repulsed by the strong form. But when they argue that all these policies are ultimately the same because they ignore people’s preferences (as demonstrated by their willingness to pay for health goods, for example), the argument doesn’t sit right with a broader audience. And then ultimately it gets ignored, because these things only really look the same if you look at them through certain ideological lenses.

That’s why I wish Easterly would take a more pragmatic approach to aid skepticism; such a form might harp on the truly coercive aspects without lumping them in with the mildly paternalistic. Condemning the truly bad things is very necessary, and folks “on the inside’ of the aid-industrial complex aren’t generally well-positioned to make those arguments publicly. However, I think people sometimes need a bit of the latter policies, the mildly paternalistic ones like giving away medicines and nudging people’s behavior — in high- and low-income countries alike. Why? Because we’re generally the same everywhere, doing what’s easiest in a given situation rather than what we might choose were the circumstances different. Having skeptics on the outside where they can rail against wrongs is incredibly important, but they must also be careful to yell at the right things lest they be ignored altogether by those who don’t share their ideological priors.

Our future selves will mock this (I hope)

Smiling people holding hands. Walking on the beach. Inexplicable doves flying through blue skies. Terrible side effects discussed cheerily by a honey-voiced narrator…. That’s right, this post is about direct-to-consumer pharmaceutical advertising.

Niam Hardimh, writing at Crooked Timber, shares one of the odd things about living in the US — for those who aren’t used to our TV:

One thing that is striking, compared with European TV, is what is advertised and how. In particular,  I don’t think you see ads for prescription medicines in Europe, certainly not in Ireland or the UK. They seem to be all over American TV.

I am particularly struck by the way these ads are made. The visuals  typically show someone having a happy and trouble-free life while using these drugs, overlaid with soothing music and a reassuringly bland voice-over. But clearly the US FDA requires advertisers to include all the small print in their ads as well.

Do you read all the known downsides of the medicines you take? Don’t…

It’s easy to become habituated to these since they’re everywhere, but it hasn’t always been that way, and in most places it still isn’t — the US and New Zealand are the only two countries that allow direct advertising of drugs. Here’s an exemplary ad for Vioxx, which was pulled off the market because it caused health problems (which Merck systematically lied about):

Ice skating. A minor celebrity. Inspiring music. They even note that “Vioxx specifically targets the Cox2 enzyme.” How many Americans can even define what an enzyme is? I’m sure consumers are more likely to remember that than the mentioned side effects (“bleeding can occur without warning”)… Other lovely examples include this other ad for Vioxx, and one for Zocor.

For more examples and some background on how the ads came to be, check out “Sick of pharmaceutical ads: here’s why they won’t go away” on io9.

09

05 2012

Obesity in the US

One of my classmates whose primary interest is not health policy posted this graph on Facebook, saying “This is stunning… so much so in fact that I’m a bit skeptical of its accuracy.”

The graph compares obesity rates by state in 1994 vs. 2008, and unfortunately it is both terrifying and accurate. (I can’t find the original source of this particular infographic, but the data is the same as on this CDC page.)

I think those of who study or work in public health have seen variations on these graphs so many times that they’ve lost some of their shock value. But this truly is an incredible shift in population health in a frighteningly short period of time. In 1994 every state had an adult population that was less than 20% obese, and many were less than 15% obese. A mere 14 years later, Colorado is the only state under 20%, and quite a few have rates over 30% — these were completely unheard of before.

I did a quick literature search, trying to understand what causal factors might be responsible for such a rapid shift. It’s a huge and challenging question, so maybe it should be unsurprising that I didn’t find an article that really stood out as the best. Still, here are three articles that I found helpful:

1. Specifically looking at childhood obesity in the US (which is different from the rates highlighted in the map above, but related): “Childhood Obesity: Trends and Potential Causes” by Anderson and Butcher (JStor PDF, ungated PDF). Their intro:

The increase in childhood obesity over the past several decades, together with the associated health problems and costs, is raising grave concern among health care professionals, policy experts, children’s advocates, and parents. Patricia Anderson and Kristin Butcher document trends in children’s obesity and examine the possible underlying causes of the obesity epidemic.

They begin by reviewing research on energy intake, energy expenditure, and “energy balance,” noting that children who eat more “empty calories” and expend fewer calories through physical activity are more likely to be obese than other children. Next they ask what has changed in children’s environment over the past three decades to upset this energy balance equation. In particular, they examine changes in the food market, in the built environment, in schools and child care settings, and in the role of parents-paying attention to the timing of these changes.

Among the changes that affect children’se nergy intake are the increasing availability of energy dense, high-calorie foods and drinkst hroughs chools. Changes in the family, particularly increasing dual-career or single-parent working families, may also have increased demand for food away from home or pre-prepared foods. A host of factors have also contributed to reductions in energy expenditure. In particular, children today seem less likely to walk to school and to be traveling more in cars than they were during the early 1970s, perhaps because of changes in the built environment. Finally, children spend more time viewing television and using computers.

Anderson and Butcher find no one factor that has led to increases in children’s obesity. Rather, many complementary changes have simultaneously increased children’s energy intake and decreased their energy expenditure. The challenge in formulating policies to address children’s obesity is to learn how best to change the environment that affects children’s energy balance.

2. On global trends: “The global obesity pandemic: shaped by global drivers and local environments” by Swinburn et al. (Here’s the PDF from Science Direct and an ungated PDF for those not at universities.) Summary:

The simultaneous increases in obesity in almost all countries seem to be driven mainly by changes in the global food system, which is producing more processed, affordable, and effectively marketed food than ever before. This passive overconsumption of energy leading to obesity is a predictable outcome of market economies predicated on consumption-based growth. The global food system drivers interact with local environmental factors to create a wide variation in obesity prevalence between populations.

Within populations, the interactions between environmental and individual factors, including genetic makeup, explain variability in body size between individuals. However, even with this individual variation, the epidemic has predictable patterns in subpopulations. In low-income countries, obesity mostly affects middle-aged adults (especially women) from wealthy, urban environments; whereas in high-income countries it affects both sexes and all ages, but is disproportionately greater in disadvantaged groups.

Unlike other major causes of preventable death and disability, such as tobacco use, injuries, and infectious diseases, there are no exemplar populations in which the obesity epidemic has been reversed by public health measures. This absence increases the urgency for evidence-creating policy action, with a priority on reduction of the supply-side drivers.

3. Finally, on methodological differences and where the trends are heading: Obesity Prevalence in the United States — Up, Down, or Sideways?(NEJM, ungated PDF). Evidently there’s some debate over whether rates are going up or have stabilized in the last few years, because different data sources say different things. Generally the NHANES data (in which people are actually measured, rather than reporting their height and weight) is the best available (and that’s what the maps above are made from). An excerpt:

One key reason for discrepancies among the estimates is a simple difference in data-collection methods. The most frequently quoted data sources are the NHANES studies of adults and children, the BRFSS for adults, and the CDC’s Youth Risk Behavior Survey (YRBS)4 for high- school students. Although sampling strategies, response rates, age discrepancies, and the wording of survey questions may account for some variability, a major factor is that in calculating the BMI, the BRFSS and YRBS rely on respondents’ self-reported heights and weights, whereas the NHANES collects measured (i.e., actual) heights and weights each year, albeit from a considerably smaller sample of the population. Since people often claim to be taller than they are and to weigh less than they actually do, we should not be surprised that obesity prevalence figures based on self-reported heights and weights are considerably lower than those based on measured data.

I would greatly appreciate any suggestions for what to read in the comments, especially links to work that tries to rigorously assess (rather than just hypothesize on) the relative import of various drivers of the increase in adult obesity.

01

05 2012

Group vs. individual uses of data

Andrew Gelman notes that, on the subject of value-added assessments of teachers, “a skeptical consensus seems to have arisen…” How did we get here?

Value-added assessments grew out of the push for more emphasis on measuring success through standardized tests in education — simply looking at test scores isn’t OK because some teachers are teaching in better schools or are teaching better-prepared students. The solution was to look at how teachers’ students improve in comparison to other teachers’ students. Wikipedia has a fairly good summary here.

Back in February New York City released (over the opposition of teachers’ unions) the value-added scores of some 18,000 teachers. Here’s coverage from the Times on the release and reactions.

Gary Rubinstein, an education blogger, has done some analysis of the data contained in the reports and published five posts so far: part 1, part 2, part 3, part 4, and part 5. He writes:

For sure the ‘reformers’ have won a battle and have unfairly humiliated thousands of teachers who got inaccurate poor ratings. But I am optimistic that this will be be looked at as one of the turning points in this fight. Up until now, independent researchers like me were unable to support all our claims about how crude a tool value-added metrics still are, though they have been around for nearly 20 years. But with the release of the data, I have been able to test many of my suspicions about value-added.

I suggest reading his analysis in full, or at least the first two parts.

For me one early take-away from this — building off comments from Gelman and others — is that an assessment might be a useful tool for improving education quality overall, while simultaneously being a very poor metric for individual performance. When you’re looking at 18,000 teachers you might be able to learn what factors lead to test score improvement on average, and use that information to improve policies for teacher education, recruitment, training, and retention. But that doesn’t mean one can necessarily use the same data to make high-stakes decisions about individual teachers.

On food deserts

Gina Kolata, writing for the New York Times, has sparked some debate with this article: “Studies Question the Pairing of Food Deserts and Obesity”. In general I often wish that science reporting focused more on how the new studies fit in with the old, rather than just the (exciting) new ones. On first reading I noticed that one study is described as having explored the association of “the type of food within a mile and a half of their homes” with what people eat.

This raised a little question mark in my mind, as I know that prior studies have often looked at distances much shorter than 1.5 miles, but it was mostly a vague hesitation. And if you didn’t know that before reading the article, then you’ve missed a major difference between the old and new results (and one that could have been easily explained). Also, describing something as “an article of faith when it’s arguably something more like “the broad conclusion draw from most most prior research“… that certainly established an editorial tone from the beginning.

Intrigued, I sent the piece to a friend (and former public health classmate) who has work on food deserts, to get a more informed reaction. I’m sharing her thoughts here (with permission) because this is an area of research that I don’t follow as closely, and her reactions helped me to situate this story in the broader literature:

1. This quote from the article is so good!

“It is always easy to advocate for more grocery stores,” said Kelly D. Brownell, director of Yale University’s Rudd Center for Food Policy and Obesity, who was not involved in the studies. “But if you are looking for what you hope will change obesity, healthy food access is probably just wishful thinking.”

The “unhealthy food environment” has a much bigger impact on diet than the “healthy food environment”, but it’s politically more viable to work from an advocacy standpoint than a regulatory standpoint. (On that point, you still have to worry about what food is available – you can’t just take out small businesses in impoverished neighborhoods and not replace it with anything.)

2. The article is too eager to dismiss the health-food access relationship. There’s good research out there, but there’s constant difficulty with tightening methods/definitions and deciding what to control for. The thing that I think is really powerful about the “food desert” discourse is that it opens doors to talk about race, poverty, community, culture, and more. At the end of the day, grocery stores are good for low-income areas because they bring in money and raise property values. If the literature isn’t perfect on health effects, I’m still willing to advocate for them.

3. I want to know more about the geography of the study that found that low-income areas had more grocery stores than high-income areas. Were they a mix of urban, peri-urban, and rural areas? Because that’s a whole other bear. (Non-shocker shocker: rural areas have food deserts… rural poverty is still a problem!)

4. The article does a good job of pointing to how difficult it is to study this. Hopkins (and the Baltimore Food Czar) are doing some work with healthy food access scores for neighborhoods. This would take into account how many healthy food options there are (supermarkets, farmers’ markets, arabers, tiendas) and how many unhealthy food options there are (fast food, carry out, corner stores).

5. The studies they cite are with kids, but the relationship between food insecurity (which is different, but related to food access) and obesity is only well-established among women. (This, itself, is not talked about enough.) The thinking is that kids are often “shielded” from the effects of food insecurity by their mothers, who eat a yo-yo diet depending on the amount of food in the house.

My friend also suggested the following articles for additional reading:

Name that quote

I’m reading Evolving Economics, a highly-regarded history of economic thought by Agnar Sandmo. I thought one tidbit early on was quite interesting: it comes in the course of a discussion of a once-common method of charging tolls based on the weight of carriages. Sandmo quotes an economist who recommended different rates for luxury versus other transport.

Thus, “…the indolence and vanity of the rich is made to contribute in a very easy manner to the relief of the poor, by rendering cheaper the transportation of heavy goods to all the different parts of the country.”

Who said that? Answer below the fold…

Read the rest of this entry →

20

04 2012

Fluoride in New Jersey

I saw this poster at a bus stop on campus a couple weeks ago:

If you can’t read it, the title reads: “Stop the New Jersey Public Water Supply Fluoridation Act” and it goes on to say “Fluoride is a toxic chemical even in the smallest doses and when pumped into our water supply it is impossible to control the level of consumption.” (emphasis added)

I took a picture but didn’t think about it again until I saw this article on Friday: “In New Jersey, a Battle Over a Fluoridation Bill, and the Facts” (NYT) by Kate Zernike. I appreciate that she calls the fearmongering what it is — a conspiracy theory:

While 72 percent of Americans get their water from public systems that add fluoride, just 14 percent of New Jersey residents do, placing the state next to last… A bill in the Legislature would change that, requiring all public water systems in New Jersey to add fluoride to the supply. But while the proposal has won support from a host of medical groups, it has proved unusually politically charged.

Similar bills have failed in the state since 2005, under pressure from the public utilities lobby and municipalities that argue that fluoridation costs too much, environmentalists who say it pollutes the water supply, and antifluoride activists who argue that it causes cancer, lowers I.Q. and amounts to government-forced medicine.

Public health officials argue that the evidence does not support any of those arguments — and to the contrary, that fluoridating the water is the single best weapon in fighting tooth decay, the most prevalent disease among children.

But they also say they are fighting a proliferation of misleading information. While conspiracy theories about fluoride in public water supplies have circulated since the early days of the John Birch Society, they now thrive online, where anyone, with a little help from Google, can suddenly become a medical authority.

The whole article is worth a read. I think it’s a pretty good journalistic take on a charged issue that is a political controversy but not a scientific one. It gives some context as to why people are against it — a few misleading studies amplified by word of mouth and the Internet — but also emphasizes which side the evidence base (overwhelmingly) backs up.

Further, there are some echoes here of the anti-vaccine movement,  in that a move to reduce the threshold of acceptable fluoride levels  by HHS was taken to be an acknowledgment that the worst fears of the fluoridation foes were vindicated. That parallels how any mention of efforts to improve vaccine safety (a good thing) is misshapen by antivaccine activists into an acknowledgment that their theories have been vindicated. In short, I’m looking forward to Seth Mnookin‘s take on all this.

06

03 2012

The US health care non-system

I spent much of yesterday thinking about the past, present, and future of the American health care system. I’ve largely chosen classes with an international or methodological focus so this was a bit of a departure from my normal fare. In one day I finished up some readings on health reform, wrote a brief paper speculating on what US healthcare will look like in 2030, attended a talk by Uwe Reinhardt largely based on this paper (PDF), and went to a three hour lecture on US health care (part of a class on the economics of the US welfare state).

It’s a mammoth subject, and there are many bloggers who write exclusively about domestic health policy — the guys at the Incidental Economist have smart stuff to say on it every day. There’s so much to be said and done even on the somewhat narrowed subject of the Affordable Care Act (ie, “ObamaCare”).

But that’s not what keeps popping into my head.What keeps getting reinforced is how our system really isn’t a system at all, but a weird conglomeration of lots of different approaches for various fragments of our society that emerged for quirky historical and political reasons. I found this description — from a report comparing various industrialized countries’ systems — humorously understated: “The U.S. does not have a ‘health system,’ but rather a variety of private and public institutions and programs that regulate, finance, and deliver care.” (source)

Paul Starr’s classic Social Transformation of American Medicine is a good start for trying to understand how we got to the ‘variety’ we have today.  The end result is that it doesn’t serve very many people well at all. The US is a great place to get the most advanced care if you can afford it, but even then you’re going to pay a lot more for it. For the non-wealthy the expenses are amplified and we end up rationing care by ability to pay. By pretty much every standard other than innovation (ie, including the delivery of that innovation to those who really need it, not just those who can pay) the US falls dreadfully short. We get poor life expectancy, magnified inequalities, and spending that’s roughly twice as much per person as in any other wealthy country.

Ironically, whether the Affordable Care Act goes into effect in 2014 depends largely on whether Obama gets reelected, and whether Obama gets reelected or not depends largely on what the unemployment rate does between now and November. So the future of the US health system depends in a very real way on fluctuations in the economy over the next eight months, and no one really understand that well at all.

If you’re just looking at the trajectory of the American health system the ACA is a major reform, even a fundamental one.  It will do (and has already started to do) a lot of good things, but I’m skeptical that it will do all that much to fix costs or shift our focus to public health —prevention over treatment. There are a lot of good small fixes in there, but nothing revolutionary when you compare us to other countries.

And this is why I find domestic health policy profoundly depressing. It’s why I’ve chosen to focus more on international health than domestic politics. In international health I think the prospects for witnessing and contributing to massive, heartening, orders-of-magnitude positive change in my professional lifetime are quite real. On US health policy, I’m less optimistic. My friend and classmate Jesse Singal wrote a description of the US health system — in the context of astonishingly ridiculous remarks by some conservatives on contraception — that I think about sums it up:  “…our medical system is an octopus riding a donkey riding a skateboard into a sadness quarry.”

01

03 2012

Platform evaluation

Cesar Victora,  Bob Black,  Ties Boerma, and Jennifer Bryce (three of the four are with the Hopkins Department of International Health and I took a course with Prof Bryce) wrote this article in The Lancet in January 2011: “Measuring impact in the Millennium Development Goal era and beyond: a new approach to large-scale effectiveness evaluations.” The abstract:

Evaluation of large-scale programmes and initiatives aimed at improvement of health in countries of low and middle income needs a new approach. Traditional designs, which compare areas with and without a given programme, are no longer relevant at a time when many programmes are being scaled up in virtually every district in the world. We propose an evolution in evaluation design, a national platform approach that: uses the district as the unit of design and analysis; is based on continuous monitoring of different levels of indicators; gathers additional data before, during, and after the period to be assessed by multiple methods; uses several analytical techniques to deal with various data gaps and biases; and includes interim and summative evaluation analyses. This new approach will promote country ownership, transparency, and donor coordination while providing a rigorous comparison of the cost-effectiveness of different scale-up approaches.