Archive for the ‘international development’Category

Several job opps

Some other good places to look for jobs: mHealth student Google group and the African Development Jobs blog.

04

06 2013

"What is wrong (and right) in economics?"

Economist Dani Rodrik has a great essay up on his website on what’s good and bad about economics. Here’s a bit on the relationship between trade policy and growth:

I remember well the reception I got when I presented my paper (with Francisco Rodriguez) on the empirics of trade policy and growth. The literature had filled up with extravagant claims about the effect of trade liberalization on economic growth. What we showed in our paper is that the research to date could not support those claims. Neither the theoretical nor empirical literature indicated there is a robust, predictable, and quantitatively large effect of trade liberalization on growth. We were simply stating what any well-trained economist should have known. Nevertheless, the paper was highly controversial. One of my Harvard colleagues asked me in the Q&A session: “why are you doing this?” It was a stunning question. It was as if knowledge of a certain kind was dangerous.

There’s a lot of good material in there about what economics is and isn’t, and how to do it better.  I had forgotten that Rodrik studied at Princeton, so was pleasantly surprised by this:

However, contemporary economics in North America has one great weakness, and that is the excessive focus on methods at the expense of breadth in terms of social and historical perspective. PhD programs now train applied mathematicians and statisticians rather than real economists. To become a true economist, you need to do all sorts of reading – from history, sociology, and political science among other disciplines – that you are never required to do as a graduate student. The best economists today find a way of filling this gap in their education. I consider myself very lucky that I was a political science major and did a master’s in public affairs (as it is called at Princeton) before I turned to economics. I say lucky, because some of my best work – by my judgement, at least – was stimulated by questions or arguments I encountered outside of neoclassical economics.

07

05 2013

(Not) knowing it all along

David McKenzie is one of the guys behind the World Bank’s excellent and incredibly wonky Development Impact blog. He came to Princeton to present on a new paper with Gustavo Henrique de Andrade and Miriam Bruhn, “A Helping Hand or the Long Arm of the Law? Experimental evidence on what governments can do to formalize firms” (PDF). The subject matter — trying to get small, informal companies to register with the government — is outside my area of expertise. But I thought there were a couple methodologically interesting bits:

First, there’s an interesting ethical dimension, as one of their several interventions tested was increasing the likelihood that a firm would be visited by a government inspector (i.e., that the law would be enforced). From page 10:

In particular, if a firm owner were interviewed about their formality status, it may not be considered ethical to then use this information to potentially assign an inspector to visit them. Even if it were considered ethical (since the government has a right to ask firm owners about their formality status, and also a right to conduct inspections), we were still concerned that individuals who were interviewed in a baseline survey and then received an inspection may be unwilling to respond to a follow-up. Therefore a listing stage was done which did not involve talking to the firm owner.

In other words, all their baseline data was collected without actually talking to the firms they were studying — check out the paper for more on how they did that.

Second, they did something that could (and maybe should) be incorporated into many evaluations with relative ease. Because findings often seem obvious after we hear them, McKenzie et al. asked the government staff whose program they were evaluating to estimate what the impact would be before the results were in. Here’s that section (emphasis added):

A standard question with impact evaluations is whether they deliver new knowledge or merely formally confirm the beliefs that policymakers already have (Groh et al, 2012). In order to measure whether the results differ from what was anticipated, in January 2012 (before any results were known) we elicited the expectations of the Descomplicar [government policy] team as to what they thought the impacts of the different treatments would be. Their team expected that 4 percent of the control group would register for SIMPLES [the formalization program] between the baseline and follow-up surveys. We see from Table 7 that this is an overestimate…

They then expected the communication only group to double this rate, so that 8 percent would register, that the free cost treatment would lead to 15 percent registering, and that the inspector treatment would lead to 25 percent registering…. The zero or negative impacts of the communication and free cost treatments therefore are a surprise. The overall impact of the inspector treatment is much lower than expected, but is in line with the IV estimates, suggesting the Descomplicar team have a reasonable sense of what to expect when an inspection actually occurs, but may have overestimated the amount of new inspections that would take place. Their expectation of a lack of impact for the indirect inspector treatment was also accurate.

This establishes exactly what in the results was a surprise and what wasn’t. It might also make sense for researchers to ask both the policymakers they’re working with and some group of researchers who study the same subject to give such responses; it would certainly help make a case for the value of (some) studies.

Rearranging the malarial deck chairs?

A friend sent this link to me, highlighting a critical comment about the future of the World Health Organization, in the context of the World Malaria Report 2012. Here’s an excerpt of the comment by William Jobin:

Their 2012 Annual Report is a very disturbing report from WHO, for at least two reasons:

1. Their program is gradually falling apart, and they offer no way to refocus, no strategy for dealing with the loss in funding, nor the brick wall of drug and biocide resistance which is just down the road. There is a label for people who keep doing the same thing, but expect different results. Do you remember what it is?

2. Because the entire top management of WHO consists of physicians, they have no idea of the opportunities they are missing for additional funding and for additional methods to add to their chemically-oriented strategy…

Concluding with:

I am not sure WHO has much of a future, nor does the UN system itself, after their failure to prevent the wars in Libya and Syria. But as long as the UN and WHO continue to operate, they must refocus their approach to face the reality of a rapidly declining budget from UN sources. Instead, I see them just re-arranging the deck chairs on the Titanic.

My friend said, “I wish these comments (and issues with the WHO and UN) were more publicised! This is not the first time I am hearing of such issues with the WHO and its demise.” I’ve certainly heard similar sentiments about the WHO from classmates and professors, but it seems there’s much less open discussion than you might expect. I’d welcome discussion in the comments…

28

03 2013

Note to job seekers

The first question I’ve had in several recent job interviews/conversations was “do you speak French?” (I don’t.) Not that it’s impossible to find work if you don’t — but it certainly seems to be a major asset. If you want to work in global health, take note.

20

03 2013

On regressions and thinking

Thesis: thinking quantitatively changes the way we frame and answer questions in ways we often don’t notice.

James Robinson, of Acemoglu and Robinson fame (ie, Why Nations Fail@whynationsfailColonial Origins; Reversal of Fortune, and so forth), gave a talk at Princeton last week. It was a good talk, mostly about Why Nations Fail. My main thought during his presentation was that it’s simply very difficult to develop a parsimonious theory that covers something as complicated as the long-term economic and political development of the entire world! As Robinson said (quoting someone else), in social science you can say “more and more about less and less, or less and less about more and more.”

The talk was followed by some great discussion where several of the tougher questions came from sociologists and political economists. I think it’s safe to say that a lot of the skepticism of the Why Nations Fail thesis is centered around the beef that East Asian economics, and especially China, don’t fit neatly into it. A&R argue here on their blog — not to mention in their book, which I’ve alas only had time to skim — that China is not an exception to their theory, but I think that impression is still fairly widespread.

But my point isn’t about the extent to which China fits into the theory (that’s another debate altogether); it’s about what it means if or when China doesn’t fit into the theory. Is that a major failure or a minor one?  I think different answers to that question are ultimately rooted in a difference of methodological cultures in the social science world.

As social science becomes more quantitative, our default method for thinking about a subject can shift, and we might not even notice that it’s happening. For example, if your main form of evidence for a theory is a series of cross-country regressions, then you automatically start to think of countries as the unit of analysis, and, importantly, as being more or less equally weighted. There are natural and arguably inevitable reasons why this will be the case: states are the clearest politicoeconomic units, and even if they weren’t they’re simply the unit for which we have the most data. While you might (and almost certainly should!) weight your data points by population if you were looking at something like health or individual wealth or well-being, it makes less sense when you’re talking about country-level phenomena like economic growth rates. So you end up seeing a lot of arguments made with scatterplots of countries and fitted lines — and you start to think that way intuitively.

When we switch back to narrative forms of thinking, this is less true: I think we all agree that all things being equal a theory that explains everything except Mauritius is better than a theory that explains everything except China. But it’s a lot harder to think intuitively about these things when you have a bunch of variables in play at the same time, which is one reason why multiple regressions are so useful. And between the extremes of weighting all countries equally and weighting them by population are a lot of potentially more reasonable ways of balancing the two concerns, that unfortunately would involve a lot of arbitrary decisions regarding weighting…

This is a thought I’ve been stewing on for a while, and it’s reinforced whenever I hear the language of quantitative analysis working its way into qualitative discussions — for instance, Robinson said at one point that “all that is in the error term,” when he wasn’t actually talking about a regression. I do this sort of thing too, and don’t think there’s anything necessarily wrong with it — until there is.  When questioned on China, Robinson answered briefly and then transitioned to talking about the Philippines, rather than just concentrating on China. If the theory doesn’t explain China (at least to the satisfaction of many), a nation of 1.3 billion, then explaining a country of 90 million is less impressive. How impressive you find an argument depends in part on the importance you ascribe to the outliers, and that depends in part on whether you were trained in the narrative way of thinking, where huge countries are hugely important, or the regression way of thinking, where all countries are equally important units of analysis.

[The first half of my last semester of school is shaping up to be much busier than expected — my course schedule is severely front-loaded — so blogging has been intermittent. Thus I’ll try and do more quick posts like this rather than waiting for the time to flesh out an idea more fully.]

25

02 2013

Growth and stagnation in global health funding

Amanda Glassman of CGD shares the graph below from the latest IHME “Financing Global Health” report, which tells the top-line story from the report in one neat picture:

This year’s report is subtitled, “The End of the Golden Age?” Maybe. I’d start with Amanda’s analysis here, then dive into the report overview [PDF].

18

02 2013

Alwyn Young just broke your regression

Alwyn Young — the same guy whose paper carefully accounting for growth in East Asian was popularized by Krugman and sparked an enormous debate — has been circulating a paper on African growth rates. Here’s the 2009 version (PDF) and October 2012 version. The abstract of the latter paper:

Measures of real consumption based upon the ownership of durable goods, the quality of housing, the health and mortality of children, the education of youth and the allocation of female time in the household indicate that sub-Saharan living standards have, for the past two decades, been growing about 3.4 to 3.7 percent per annum, i.e. three and a half to four times the rate indicated in international data sets. (emphasis added)

The Demographic and Health Surveys are large-scale nationally-representative surveys of health, family planning, and related modules that tend to ask the same questions across different countries and over large periods of time. They have major limitations, but in the absence of high-quality data from governments they’re often the best source for national health data. The DHS doesn’t collect much economic data, but they do ask about ownership of certain durable goods (like TVs, toilets, etc), and the answers to these questions are used to construct a wealth index that is very useful for studies of health equity — something I’m taking advantage of in my current work. (As an aside, this excellent report from Measure DHS (PDF) describes the history of the wealth index.)

What Young has done is to take this durable asset data from many DHS surveys and try to estimate a measure of GDP growth from actually-measured data, rather than the (arguably) sketchier methods typically used to get national GDP numbers in many African countries. Not all countries are represented at any given point in time in the body of DHS data, which is why he ends up with a very-unbalanced panel data set for “Africa,” rather than being able to measure growth rates in individual countries. All the data and code for the paper are available here.

Young’s methods themselves are certain to spark ongoing debate (see commentary and links from Tyler Cowen and Chris Blattman), so this is far from settled — and may well never be. The takeaway is probably not that Young’s numbers are right so much as that there’s a lot of data out there that we shouldn’t trust very much, and that transparency about the sources and methodology behind data, official or not, is very helpful. I just wanted to raise one question: if Young’s data is right, just how many published papers are wrong?

There is a huge literature on cross-country growth ‘s empirics. A Google Scholar search for “cross-country growth Africa” turns up 62,400 results. While not all of these papers are using African countries’ GDPs as an outcome, a lot of them are. This literature has many failings which have been duly pointed out by Bill Easterly and many others, to the extent that an up-and-coming economist is likely to steer away from this sort of work for fear of being mocked. Relatedly, in Acemoglu and Robinson’s recent and entertaining take-down of Jeff Sachs, one of their insults criticisms is that Sachs only knows something because he’s been running “kitchen sink growth regressions.”

Young’s paper just adds more fuel to that fire. If African GDP growth has been 3 1/2 to 4 times greater than the official data says, then every single paper that uses the old GDP numbers is now even more suspect.

On deworming

GiveWell’s Alexander Berger just posted a more in-depth blog review of the (hugely impactful) Miguel and Kremer deworming study. Here’s some background: the Cochrane reviewGivewell’s first response to it, and IPA’s very critical response.

I’ve been meaning to blog on this since the new Cochrane review came out, but haven’t had time to do the subject justice by really digging into all the papers. So I hope you’ll forgive me for just sharing the comment I left at the latest GiveWell post, as it’s basically what I was going to blog anyway:

Thanks for this interesting review — I especially appreciate that the authors [Miguel and Kremer] shared the material necessary for you [GiveWell] to examine their results in more depth, and that you talk through your thought process.

However, one thing you highlighted in your post on the new Cochrane review that isn’t mentioned here, and which I thought was much more important than the doubts about this Miguel and Kremer study, was that there have been so many other studies that did not find large effect on health outcomes! I’ve been meaning to write a long blog post about this when I really have time to dig into the references, but since I’m mid-thesis I’ll disclaim that this quick comment is based on recollection of the Cochrane review and your and IPA’s previous blog posts, so forgive me if I misremember something.

The Miguel and Kremer study gets a lot of attention in part because it had big effects, and in part because it measured outcomes that many (most?) other deworming studies hadn’t measured — but it’s not as if we believe these outcomes to be completely unrelated. This is a case where what we believe the underlying causal mechanism for the social effects to be is hugely important. For the epidemiologists reading, imagine this as a DAG (a directed acyclic graph) where the mechanism is “deworming -> better health -> better school attendance and cognitive function -> long-term social/economic outcomes.” That’s at least how I assume the mechanism is hypothesized.

So while the other studies don’t measure the social outcomes, it’s harder for me to imagine how deworming could have a very large effect on school and social/economic outcomes without first having an effect on (some) health outcomes — since the social outcomes are ‘downstream’ from the health ones. Maybe different people are assuming that something else is going on — that the health and social outcomes are somehow independent, or that you just can’t measure the health outcomes as easily as the social ones, which seems backwards to me. (To me this was the missing gap in the IPA blog response to GiveWell’s criticism as well.)

So continuing to give so much attention to this study, even if it’s critical, misses what I took to be the biggest takeaway from that review — there have been a bunch of studies that showed only small effects or none at all. They were looking at health outcomes, yes, but those aren’t unrelated to the long-term development, social, and economic effects. You [GiveWell] try to get at the external validity of this study by looking for different size effects in areas with different prevalence, which is good but limited. Ultimately, if you consider all of the studies that looked at various outcomes, I think the most plausible explanation for how you could get huge (social) effects in the Miguel Kremer study while seeing little to no (health) effects in the others is not that the other studies just didn’t measure the social effects, but that the Miguel Kremer study’s external validity is questionable because of its unique study population.

(Emphasis added throughout)

 

Someone should study this: Addis housing edition

Attention development economists and any other researchers who have an interest in urban or housing policy in low-income countries:

My office in Addis has about 25 folks working in it, and we have a daily lunch pool where we pay in 400 birr a month (about 22 USD) to cover costs and all get to eat Ethiopian food for lunch every day. It’s been a great way to get to know my coworkers — my work is often more solitary: editing, writing, and analyzing data — and an even better way to learn about a whole variety of issues in Ethiopia.

addis construction

Addis construction site (though not probably not government condos)

The conversation is typically in Amharic and mine is quite limited, so I’m lucky if I can figure out the topic being discussed.  [I usually know if they’re talking about work because so many NGO-speak words aren’t translated, for example: “amharic amharic amharic Health Systems Strengthening amharic amharic…“] But folks will of course translate things as needed.  One observation is that certain topics affect their daily lives a lot, and thus come up over and over again at lunch.

One subject that has come up repeatedly is housing. Middle class folks in Addis Ababa feel the housing shortage very acutely. Based on our conversations it seems the major limitation is in getting credit to buy or build a house.

The biggest source of good housing so far has been government-constructed condominiums, for which you pay a certain (I’m not sure how much) percentage down and then make payments over the years. (The government will soon launch a new “40/60 scheme” to which many folks are looking forward, in which anyone who can make a 40% down payment on a house will get a government mortgage for the remaining 60%.)

When my coworkers first mentioned that the government will offer the next round of condominiums by a public lottery, my thought was “that will solve someone’s identification problem!” A large number of people — many thousands — have registered for the government lottery. I believe you have to meet a certain wealth or income threshold (i.e., be able to make the down payment), but after that condo eligibility will be determined randomly. I think that — especially if someone organizes the study prior to the lottery — this could yield very useful results on the impact of urban housing policy.

How (and how much) do individuals and families benefit from access to better housing? Are there changes in earnings, savings, investments? Health outcomes? Children’s health and educational outcomes? How does it affect political attitudes or other life choices? It could also be an opportunity to study migration between different neighborhoods, amongst many other things.

A Google Scholar search for Ethiopia housing lottery turns up several mentions, but (in my very quick read) no evaluations taking advantage of the randomization. (I can’t access this recent article in an engineering journal, but from the abstract assume that it’s talking about a different kind of evaluation.) So, someone have at it? It’s just not that often that large public policy schemes are randomized.

06

12 2012