Archive for the ‘economics’Category

Formalizing corruption: US medical system edition

Oh, corruption. It interferes with so many aspects of daily life, adding time to the simplest daily tasks, costing more money, and — often the most frustrating aspect — adding huge doses of uncertainty. That describes life in many low-income, high-corruption countries, leading to many a conversation with friends about comparisons with the United States and other wealthy countries. How did the US “solve” corruption?

I’ve heard (and personally made) the argument that the US reduced corruption at least in part by formalizing it; by channeling the root of corruption, a sort of rent-seeking on a personal level, to rent-seeking on an institutional level. The US political and economic system has evolved such that some share of any wealth created is channeled into the pockets of a political and economic elite who benefit from the system and in turn reinforce it. That unproductively-channeled share of wealth is simultaneously a) probably smaller than the share of wealth lost to corruption in most developing countries, b) still large enough to head off — along with the threat of more effective prosecution — at least some more overt corruption, and c) still a major drain on society.

An example: Elisabeth Rosenthal profiles medical tourism in an impressive series in the New York Times. In part three of the series, an American named Michael Shopenn travels to Belgium to get a hip replacement. Why would he need to? Because health economics in the US is less a story of free markets and  more a story of political capture by medical interests, including technology and pharmaceutical companies, physicians’ groups, and hospitals:

Generic or foreign-made joint implants have been kept out of the United States by trade policy, patents and an expensive Food and Drug Administration approval process that deters start-ups from entering the market. The “companies defend this turf ferociously,” said Dr. Peter M. Cram, a physician at the University of Iowa medical school who studies the costs of health care.

Though the five companies make similar models, each cultivates intense brand loyalty through financial ties to surgeons and the use of a different tool kit and operating system for the installation of its products; orthopedists typically stay with the system they learned on. The thousands of hospitals and clinics that purchase implants try to bargain for deep discounts from manufacturers, but they have limited leverage since each buys a relatively small quantity from any one company.

In addition, device makers typically require doctors’ groups and hospitals to sign nondisclosure agreements about prices, which means institutions do not know what their competitors are paying. This secrecy erodes bargaining power and has allowed a small industry of profit-taking middlemen to flourish: joint implant purchasing consultants, implant billing companies, joint brokers. There are as many as 13 layers of vendors between the physician and the patient for a hip replacement, according to Kate Willhite, a former executive director of the Manitowoc Surgery Center in Wisconsin.

If this system existed in another country we wouldn’t hesitate to call it corrupt, and to note that it actively hurts consumers. It should be broken up by legislation for the public good, but instead it’s protected by legislators who are lobbied by the industry and by doctors who receive kickbacks, implicit and explicit. Contrast that with the Belgian system:

His joint implant and surgery in Belgium were priced according to a different logic. Like many other countries, Belgium oversees major medical purchases, approving dozens of different types of implants from a selection of manufacturers, and determining the allowed wholesale price for each of them, for example. That price, which is published, currently averages about $3,000, depending on the model, and can be marked up by about $180 per implant. (The Belgian hospital paid about $4,000 for Mr. Shopenn’s high-end Zimmer implant at a time when American hospitals were paying an average of over $8,000 for the same model.)

“The manufacturers do not have the right to sell an implant at a higher rate,” said Philip Boussauw, director of human resources and administration at St. Rembert’s, the hospital where Mr. Shopenn had his surgery. Nonetheless, he said, there was “a lot of competition” among American joint manufacturers to work with Belgian hospitals. “I’m sure they are making money,” he added.

It’s become a cliche to compare the US medical system to European ones, but those comparisons are made because it’s hard to realize just how systematically corrupt — and expensive, as a result — the US system is without comparing it to ones that do a better job of channeling the natural profit-seeking goals of individuals and companies towards the public good. (For the history of how we got here, Paul Starr is a good place to start.)

The usual counterargument for protecting such large profit margins in the US is that they drive innovation, which is true but only to an extent. And for the implants industry that argument is much less compelling since many of the newer, “innovative” products have proved somewhere between no better and much worse in objective tests.

The Times piece is definitely worth a read. While I generally prefer the formalized corruption to the unformalized version, I’ll probably share this article with friends — in Nigeria, or Ethiopia, or wherever else the subject comes up next.

Aug

05

2013

Uninformative paper titles: “in Africa”

When I saw a new NBER working paper titled “Disease control, demographic change and institutional development in Africa” (PDF) pop up in the NBER RSS feed I thought the title sounded interesting, so I downloaded the paper to peruse later. Then today the new-ish (and great!) blog Cherokee Gothic highlighted the same paper in a post, and I finally took a look.

Unfortunately the paper title is rather uninformative, as the authors only used data from Burkina Faso. Sure, economics papers tend to have bigger, less formal titles than papers in some other fields, but I think this is particularly unhelpful. There are enough search frictions in finding applicable literature on any given topic that it helps to be somewhat more precise.

For reference, here’s Burkina Faso:

And here’s Africa:

Not the same.

It’s unclear from the data and arguments presented how these results — for a regional disease control program, but only using data from Burkina Faso — might generalize to the quite diverse disease environments, demographic trends, and institutional histories of various African countries. The paper doesn’t answer or even give much grounds for speculation on whether onchocerciasis or other disease control programs would yield similar results in countries as diverse as (for example) Senegal, Ethiopia, Uganda, and Angola.

A quick thought experiment: Virginia’s population is about 1.5% of the total population of North America, just as Burkina Faso’s population is about 1.5% of the total population on Africa. Can you imagine someone writing a paper on health and institutions using data from Virginia and titling that paper “Health and institutions in North America”? Or writing a paper on Vietnamese history and titling it “A history of Asia”? Probably not.

Jul

27

2013

On economic history

Here’s an excerpt from Peter Temin’s “The Rise and Fall of Economic History at MIT.” (PDF, via Mankiw)

What is the cost of not having economic history at MIT? It can be seen in Acemoglu and Robinson, Why Nations Fail (2012). This is a deservedly successful popular book, making a simple and strong point that the authors made originally at the professional level over a decade before (Acemoglu, Johnson and Robinson, 2001). They assert that countries can be “ruled by a narrow elite that have [sic] organized society for their own benefit at the expense of the vast mass of people” or can have “a revolution that transformed the politics and thus the economics of the nation … to expand their economic opportunities (Acemoglu and Robinson, 2012, pp. 3-4).”

The book is not however good economic history. It is an example of Whig history in which good policies make for progress and bad policies preclude it. Only transitions from bad to good are considered in this colorful but still monotonic story. The clear implication is that if countries can copy the policies of English-speaking countries, they will prosper. No consideration is given to Britain’s economic problems over the past half-century or of Australia’s relative decline for a century.

His take on the US is also rather provocative – worth a read.

This seems like a good time to recommend Evolving Economics, a history of economics that I found helpful for supplementing my crash-course introduction to the field. It’s quite dense (I still haven’t been able to read it straight through) but was a good resource for looking up particular scholars, and for understanding the path-dependency and personalities behind the development of economics.

Jul

05

2013

“What is wrong (and right) in economics?”

Economist Dani Rodrik has a great essay up on his website on what’s good and bad about economics. Here’s a bit on the relationship between trade policy and growth:

I remember well the reception I got when I presented my paper (with Francisco Rodriguez) on the empirics of trade policy and growth. The literature had filled up with extravagant claims about the effect of trade liberalization on economic growth. What we showed in our paper is that the research to date could not support those claims. Neither the theoretical nor empirical literature indicated there is a robust, predictable, and quantitatively large effect of trade liberalization on growth. We were simply stating what any well-trained economist should have known. Nevertheless, the paper was highly controversial. One of my Harvard colleagues asked me in the Q&A session: “why are you doing this?” It was a stunning question. It was as if knowledge of a certain kind was dangerous.

There’s a lot of good material in there about what economics is and isn’t, and how to do it better.  I had forgotten that Rodrik studied at Princeton, so was pleasantly surprised by this:

However, contemporary economics in North America has one great weakness, and that is the excessive focus on methods at the expense of breadth in terms of social and historical perspective. PhD programs now train applied mathematicians and statisticians rather than real economists. To become a true economist, you need to do all sorts of reading – from history, sociology, and political science among other disciplines – that you are never required to do as a graduate student. The best economists today find a way of filling this gap in their education. I consider myself very lucky that I was a political science major and did a master’s in public affairs (as it is called at Princeton) before I turned to economics. I say lucky, because some of my best work – by my judgement, at least – was stimulated by questions or arguments I encountered outside of neoclassical economics.

May

07

2013

(Not) knowing it all along

David McKenzie is one of the guys behind the World Bank’s excellent and incredibly wonky Development Impact blog. He came to Princeton to present on a new paper with Gustavo Henrique de Andrade and Miriam Bruhn, “A Helping Hand or the Long Arm of the Law? Experimental evidence on what governments can do to formalize firms” (PDF). The subject matter — trying to get small, informal companies to register with the government — is outside my area of expertise. But I thought there were a couple methodologically interesting bits:

First, there’s an interesting ethical dimension, as one of their several interventions tested was increasing the likelihood that a firm would be visited by a government inspector (i.e., that the law would be enforced). From page 10:

In particular, if a firm owner were interviewed about their formality status, it may not be considered ethical to then use this information to potentially assign an inspector to visit them. Even if it were considered ethical (since the government has a right to ask firm owners about their formality status, and also a right to conduct inspections), we were still concerned that individuals who were interviewed in a baseline survey and then received an inspection may be unwilling to respond to a follow-up. Therefore a listing stage was done which did not involve talking to the firm owner.

In other words, all their baseline data was collected without actually talking to the firms they were studying — check out the paper for more on how they did that.

Second, they did something that could (and maybe should) be incorporated into many evaluations with relative ease. Because findings often seem obvious after we hear them, McKenzie et al. asked the government staff whose program they were evaluating to estimate what the impact would be before the results were in. Here’s that section (emphasis added):

A standard question with impact evaluations is whether they deliver new knowledge or merely formally confirm the beliefs that policymakers already have (Groh et al, 2012). In order to measure whether the results differ from what was anticipated, in January 2012 (before any results were known) we elicited the expectations of the Descomplicar [government policy] team as to what they thought the impacts of the different treatments would be. Their team expected that 4 percent of the control group would register for SIMPLES [the formalization program] between the baseline and follow-up surveys. We see from Table 7 that this is an overestimate…

They then expected the communication only group to double this rate, so that 8 percent would register, that the free cost treatment would lead to 15 percent registering, and that the inspector treatment would lead to 25 percent registering…. The zero or negative impacts of the communication and free cost treatments therefore are a surprise. The overall impact of the inspector treatment is much lower than expected, but is in line with the IV estimates, suggesting the Descomplicar team have a reasonable sense of what to expect when an inspection actually occurs, but may have overestimated the amount of new inspections that would take place. Their expectation of a lack of impact for the indirect inspector treatment was also accurate.

This establishes exactly what in the results was a surprise and what wasn’t. It might also make sense for researchers to ask both the policymakers they’re working with and some group of researchers who study the same subject to give such responses; it would certainly help make a case for the value of (some) studies.

Apr

04

2013

On regressions and thinking

Thesis: thinking quantitatively changes the way we frame and answer questions in ways we often don’t notice.

James Robinson, of Acemoglu and Robinson fame (ie, Why Nations Fail@whynationsfailColonial Origins; Reversal of Fortune, and so forth), gave a talk at Princeton last week. It was a good talk, mostly about Why Nations Fail. My main thought during his presentation was that it’s simply very difficult to develop a parsimonious theory that covers something as complicated as the long-term economic and political development of the entire world! As Robinson said (quoting someone else), in social science you can say “more and more about less and less, or less and less about more and more.”

The talk was followed by some great discussion where several of the tougher questions came from sociologists and political economists. I think it’s safe to say that a lot of the skepticism of the Why Nations Fail thesis is centered around the beef that East Asian economics, and especially China, don’t fit neatly into it. A&R argue here on their blog — not to mention in their book, which I’ve alas only had time to skim — that China is not an exception to their theory, but I think that impression is still fairly widespread.

But my point isn’t about the extent to which China fits into the theory (that’s another debate altogether); it’s about what it means if or when China doesn’t fit into the theory. Is that a major failure or a minor one?  I think different answers to that question are ultimately rooted in a difference of methodological cultures in the social science world.

As social science becomes more quantitative, our default method for thinking about a subject can shift, and we might not even notice that it’s happening. For example, if your main form of evidence for a theory is a series of cross-country regressions, then you automatically start to think of countries as the unit of analysis, and, importantly, as being more or less equally weighted. There are natural and arguably inevitable reasons why this will be the case: states are the clearest politicoeconomic units, and even if they weren’t they’re simply the unit for which we have the most data. While you might (and almost certainly should!) weight your data points by population if you were looking at something like health or individual wealth or well-being, it makes less sense when you’re talking about country-level phenomena like economic growth rates. So you end up seeing a lot of arguments made with scatterplots of countries and fitted lines — and you start to think that way intuitively.

When we switch back to narrative forms of thinking, this is less true: I think we all agree that all things being equal a theory that explains everything except Mauritius is better than a theory that explains everything except China. But it’s a lot harder to think intuitively about these things when you have a bunch of variables in play at the same time, which is one reason why multiple regressions are so useful. And between the extremes of weighting all countries equally and weighting them by population are a lot of potentially more reasonable ways of balancing the two concerns, that unfortunately would involve a lot of arbitrary decisions regarding weighting…

This is a thought I’ve been stewing on for a while, and it’s reinforced whenever I hear the language of quantitative analysis working its way into qualitative discussions — for instance, Robinson said at one point that “all that is in the error term,” when he wasn’t actually talking about a regression. I do this sort of thing too, and don’t think there’s anything necessarily wrong with it — until there is.  When questioned on China, Robinson answered briefly and then transitioned to talking about the Philippines, rather than just concentrating on China. If the theory doesn’t explain China (at least to the satisfaction of many), a nation of 1.3 billion, then explaining a country of 90 million is less impressive. How impressive you find an argument depends in part on the importance you ascribe to the outliers, and that depends in part on whether you were trained in the narrative way of thinking, where huge countries are hugely important, or the regression way of thinking, where all countries are equally important units of analysis.

[The first half of my last semester of school is shaping up to be much busier than expected -- my course schedule is severely front-loaded -- so blogging has been intermittent. Thus I'll try and do more quick posts like this rather than waiting for the time to flesh out an idea more fully.]
Feb

25

2013

Why did HIV decline in Uganda?

That’s the title of an October 2012 paper (PDF) by Marcella Alsan and David Cutler, and a longstanding, much-debated question in global health circles . Here’s the abstract:

Uganda is widely viewed as a public health success for curtailing its HIV/AIDS epidemic in the early 1990s. To investigate the factors contributing to this decline, we build a model of HIV transmission. Calibration of the model indicates that reduced pre-marital sexual activity among young women was the most important factor in the decline. We next explore what led young women to change their behavior. The period of rapid HIV decline coincided with a dramatic rise in girls’ secondary school enrollment. We instrument for this enrollment with distance to school, conditional on a rich set of demographic and locational controls, including distance to market center. We find that girls’ enrollment in secondary education significantly increased the likelihood of abstaining from sex. Using a triple-difference estimator, we find that some of the schooling increase among young women was in response to a 1990 affirmative action policy giving women an advantage over men on University applications. Our findings suggest that one-third of the 14 percentage point decline in HIV among young women and approximately one-fifth of the overall HIV decline can be attributed to this gender-targeted education policy.

This paper won’t settle the debate over why HIV prevalence declined in Uganda, but I think it’s interesting both for its results and the methodology. I particularly like the bit on using distance from schools and from market center in this way, the idea being that they’re trying to measure the effect of proximity to schools while controlling for the fact that schools are likely to be closer to the center of town in the first place.

The same paper was previously published as an NBER working paper in 2010, and it looks to me as though the addition of those distance-to-market controls was the main change since then. [Pro nerd tip: to figure out what changed between two PDFs, convert them to Word via pdftoword.com, save the files, and use the 'Compare > two versions of a document' feature in the Review pane in Word.]

Also, a tip of the hat to Chris Blattman, who earlier highlighted Alsan’s fascinating paper (PDF) on TseTse flies. I was impressed by the amount of biology in the tsetse fly paper; a level of engagement with non-economic literature that I thought was both welcome and unusual for an economics paper. Then I realized it makes sense given that the author has an MD, an MPH, and a PhD in economics. Now I feel inadequate.

Jan

03

2013

Defaults

Alanna Shaikh took a few thing I said on Twitter and expanded them into this blog post. Basically I was noting — and she in turn highlighted — that on matters of paternalism vs. choice, economists’ default is consumer choice, whereas the public health default is paternalism.

This can and does result in lousy policies from both ends — for example, see my long critique of Bill Easterly’s rejection of effective but mildly paternalistic programs due to (in my view) relying too heavily on the economists’ default position.

I was reminder of all this by a recent post on the (awesomely named) Worthwhile Canadian Initiative. The blogger, Frances Wooley, quotes a from a microeconomics textbook: ”As a budding economist, you want to avoid lines of reasoning that suggest people habitually do things that make them worse off…” Can you imagine a public health textbook including that sentence? Hah! Wooley, responded, “The problem with this argument is that it flies in the face of the abundant empirical evidence that people habitually overeat, overspend, and do other things that make them worse off.”

The historical excesses and abuses of public health are also rooted in this paternalistic streak, just as many of the absurdities of economics are rooted in its own defaults. I think most folks even in these two professions fall somewhere in between these extremes, but that a lot of disagreements (and lack of respect) between the fields stems from this fundamental difference in starting points.

(See also some related thoughts from Terence at Waylaid Dialectic that I saw after writing the initial version of this post.)

 

Dec

13

2012

Alwyn Young just broke your regression

Alwyn Young — the same guy whose paper carefully accounting for growth in East Asian was popularized by Krugman and sparked an enormous debate — has been circulating a paper on African growth rates. Here’s the 2009 version (PDF) and October 2012 version. The abstract of the latter paper:

Measures of real consumption based upon the ownership of durable goods, the quality of housing, the health and mortality of children, the education of youth and the allocation of female time in the household indicate that sub-Saharan living standards have, for the past two decades, been growing about 3.4 to 3.7 percent per annum, i.e. three and a half to four times the rate indicated in international data sets. (emphasis added)

The Demographic and Health Surveys are large-scale nationally-representative surveys of health, family planning, and related modules that tend to ask the same questions across different countries and over large periods of time. They have major limitations, but in the absence of high-quality data from governments they’re often the best source for national health data. The DHS doesn’t collect much economic data, but they do ask about ownership of certain durable goods (like TVs, toilets, etc), and the answers to these questions are used to construct a wealth index that is very useful for studies of health equity — something I’m taking advantage of in my current work. (As an aside, this excellent report from Measure DHS (PDF) describes the history of the wealth index.)

What Young has done is to take this durable asset data from many DHS surveys and try to estimate a measure of GDP growth from actually-measured data, rather than the (arguably) sketchier methods typically used to get national GDP numbers in many African countries. Not all countries are represented at any given point in time in the body of DHS data, which is why he ends up with a very-unbalanced panel data set for “Africa,” rather than being able to measure growth rates in individual countries. All the data and code for the paper are available here.

Young’s methods themselves are certain to spark ongoing debate (see commentary and links from Tyler Cowen and Chris Blattman), so this is far from settled — and may well never be. The takeaway is probably not that Young’s numbers are right so much as that there’s a lot of data out there that we shouldn’t trust very much, and that transparency about the sources and methodology behind data, official or not, is very helpful. I just wanted to raise one question: if Young’s data is right, just how many published papers are wrong?

There is a huge literature on cross-country growth ‘s empirics. A Google Scholar search for “cross-country growth Africa” turns up 62,400 results. While not all of these papers are using African countries’ GDPs as an outcome, a lot of them are. This literature has many failings which have been duly pointed out by Bill Easterly and many others, to the extent that an up-and-coming economist is likely to steer away from this sort of work for fear of being mocked. Relatedly, in Acemoglu and Robinson’s recent and entertaining take-down of Jeff Sachs, one of their insults criticisms is that Sachs only knows something because he’s been running “kitchen sink growth regressions.”

Young’s paper just adds more fuel to that fire. If African GDP growth has been 3 1/2 to 4 times greater than the official data says, then every single paper that uses the old GDP numbers is now even more suspect.

Dec

11

2012

Someone should study this: Addis housing edition

Attention development economists and any other researchers who have an interest in urban or housing policy in low-income countries:

My office in Addis has about 25 folks working in it, and we have a daily lunch pool where we pay in 400 birr a month (about 22 USD) to cover costs and all get to eat Ethiopian food for lunch every day. It’s been a great way to get to know my coworkers — my work is often more solitary: editing, writing, and analyzing data — and an even better way to learn about a whole variety of issues in Ethiopia.

addis construction

Addis construction site (though not probably not government condos)

The conversation is typically in Amharic and mine is quite limited, so I’m lucky if I can figure out the topic being discussed.  [I usually know if they're talking about work because so many NGO-speak words aren't translated, for example: "amharic amharic amharic Health Systems Strengthening amharic amharic..."] But folks will of course translate things as needed.  One observation is that certain topics affect their daily lives a lot, and thus come up over and over again at lunch.

One subject that has come up repeatedly is housing. Middle class folks in Addis Ababa feel the housing shortage very acutely. Based on our conversations it seems the major limitation is in getting credit to buy or build a house.

The biggest source of good housing so far has been government-constructed condominiums, for which you pay a certain (I’m not sure how much) percentage down and then make payments over the years. (The government will soon launch a new “40/60 scheme” to which many folks are looking forward, in which anyone who can make a 40% down payment on a house will get a government mortgage for the remaining 60%.)

When my coworkers first mentioned that the government will offer the next round of condominiums by a public lottery, my thought was “that will solve someone’s identification problem!” A large number of people — many thousands — have registered for the government lottery. I believe you have to meet a certain wealth or income threshold (i.e., be able to make the down payment), but after that condo eligibility will be determined randomly. I think that — especially if someone organizes the study prior to the lottery — this could yield very useful results on the impact of urban housing policy.

How (and how much) do individuals and families benefit from access to better housing? Are there changes in earnings, savings, investments? Health outcomes? Children’s health and educational outcomes? How does it affect political attitudes or other life choices? It could also be an opportunity to study migration between different neighborhoods, amongst many other things.

A Google Scholar search for Ethiopia housing lottery turns up several mentions, but (in my very quick read) no evaluations taking advantage of the randomization. (I can’t access this recent article in an engineering journal, but from the abstract assume that it’s talking about a different kind of evaluation.) So, someone have at it? It’s just not that often that large public policy schemes are randomized.

Dec

06

2012

“As it had to fail”

My favorite line from the Anti-Politics Machine is a throwaway. The author, James Ferguson, an anthropologist, describes a World Bank agricultural development program in Lesotho, and also — through that lens — ends up describing development programs more generally. At one point he notes that the program failed “as it had to fail” — not really due to bad intentions, or to lack of technical expertise, or lack of funds — but because failure was written into the program from the beginning. Depressing? Yes, but valuable.

I read in part because Chris Blattman keeps plugging it, and then shortly before leaving for Ethiopia I saw that a friend had a copy I could borrow. Somehow it didn’t make it onto reading lists for any of my classes for either of my degrees, though it should be required for pretty much anyone wanting to work in another culture (or, for that matter, trying to foment change in your own). Here’s Blattman’s description:

People’s main assets [in Lesotho] — cattle — were dying in downturns for lack of a market to sell them on. Households on hard times couldn’t turn their cattle into cash for school fees and food. Unfortunately, the cure turned out to be worse than the disease.

It turns out that cattle were attractive investments precisely because they were hard to liquidate. With most men working away from home in South Africa, buying cattle was the best way to keep the family saving rather than spending. They were a means for men to wield power over their families from afar.

Ferguson’s point was that development organizations attempt to be apolitical at their own risk. What’s more, he argued that they are structured to remain ignorant of the historical, political and cultural context in which they operate.

And here’s a brief note from Foreign Affairs:

 The book comes to two main conclusions. First is that the distinctive discourse and conceptual apparatus of development experts, although good for keeping development agencies in business, screen out and ignore most of the political and historical facts that actually explain Third World poverty-since these realities suggest that little can be accomplished by apolitical “development” interventions. Second, although enormous schemes like Thaba-Tseka generally fail to achieve their planned goals, they do have the major unplanned effect of strengthening and expanding the power of politically self-serving state bureaucracies. Particularly good is the discussion of the “bovine mystique,” in which the author contrasts development experts’ misinterpretation of “traditional” attitudes toward uneconomic livestock with the complex calculus of gender, cash and power in the rural Lesotho family.

The reality was that Lesotho was not really an idyllically-rural-but-poor agricultural economy, but rather a labor reserve more or less set up by and controlled by apartheid South Africa. The gulf between the actual political situation and the situation as envisioned by the World Bank — where the main problems were lack of markets and technical solutions — at the time was enormous. This lets Ferguson have a lot of fun showing the absurdities of Bank reports from the era, and once you realize what’s going on it’s quite frustrating to read how the programs turned out, and to wonder how no one saw it coming.

This contrast between rhetoric and reality is the book’s greatest strength: because the situation is absurd, it illustrates Ferguson’s points very well, that aid is inherently political, and that projects that ignore that reality have their future failure baked in from the start. But that contrast is a weakness too, as because the situation is extreme you’re left wondering just how representative the case of Lesotho really was (or is). The 1970s-80s era World Bank certainly makes a great buffoon (if not quite a villain) in the story, and one wonders if things aren’t at least a bit better today.

Either way, this is one of the best books on development I’ve read, as I find myself mentally referring to it on a regular basis. Is the rhetoric I’m reading (or writing) really how it is? Is that technical, apolitical sounding intervention really going to work? It’s made me think more critically about the role outside groups — even seemingly benevolent, apolitical ones — have on local politics. On the other hand, the Anti-Politics Machine does read a bit like it was adapted from an anthropology dissertation (it was); I wish it could get a new edition with more editing to make it more presentable. And a less ugly cover. But that’s no excuse — if you want to work in development or international health or any related field, it should be high on your reading list.

Nov

15

2012

Fugue

It took the World Bank 20 years to set up an evaluation outfit — a new paper by Michele Alacevich tells the story of how that came to pass. It’s a story about, amongst other things, the tension between academia and programs, between context-specific knowledge and generalizable lessons. The abstract:

Since its birth in 1944, the World Bank has had a strong focus on development projects. Yet, it did not have a project evaluation unit until the early 1970s. An early attempt to conceptualize project appraisal had been made in the 1960s by Albert Hirschman, whose undertaking raised high expectations at the Bank. Hirschman’s conclusions—published first in internal Bank reports and then, as a book in 1967—disappointed many at the Bank, primarily because they found it impractical. Hirschman wanted to offer the Bank a new vision by transforming the Bank’s approach to project design, project management and project appraisal. What the Bank expected from Hirschman, however, was not a revolution but an examination of the Bank’s projects and advice on how to make project design and management more measurable, controllable, and suitable for replication. The history of this failed collaboration provides useful insights on the unstable equilibrium between operations and evaluation within the Bank. In addition, it shows that the Bank actively participated in the development economics debates of the 1960s. This should be of interest for development economists today who reflect on the future of their discipline emphasizing the need for a non-dogmatic approach to development. It should also be of interest for the Bank itself, which is stressing the importance of evaluation for effective development policies. The history of the practice of development economics, using archival material, can bring new perspectives and help better understand the evolution of this discipline.

And this from the introduction:

Furthermore, the Bank all but ignored the final outcome of his project, the 1967 book, and especially disliked its first chapter…. In particular, Hirschman’s insistence on uncertainty as a structural element in the decision-making process did not fit in well with the operational drive of Bank economists and engineers.

Why’d they ignore it?

The Bank, Hirschman wrote, should avoid the “air of pat certainty” that emanated from project prospects and instead expose the uncertainties underlying them, exploring the whole range of possible outcomes. Moreover, the Bank should take into account the distributional and, more generally, the social and political effects of its lending.

It seems that one of the primary lessons of studying development economics is that many if not most of the biggest arguments you hear today already took place a generation ago. As with fashion, trends come and go, and ultimately come again. The arguments weren’t necessarily solved, they were just pushed aside when something newer and shinier came along. Even the argument against bold centrally-planned strategies — and in favor of facing up to the inherent uncertainty of complex systems — has been made before. It failed to catch on, for reasons of politics and personality. Ultimately the systems in place may not want to hear results that downplay their importance and potency the grand scheme of things. On that note it seems that if history doesn’t exactly repeat itself, it will at least continue to have some strong echoes of past debates.

Alacevich’s paper is free to download here. H/t to Andres Marroquin, who reads and shares a ridiculous number of interesting things.

Nov

14

2012

Another type of mystification

A long time ago (in years, two more than the product of 10 and the length of a single American presidential term) John Siegfried wrote this First Lesson in Econometrics (PDF). It starts with this:

Every budding econometrician must learn early that it is never in good taste to express the sum of the two quantities in the form: “1 + 1 = 2″.

… and just goes downhill from there. Read it.

(I wish I remembered where I first saw this so I could give them credit.)

Sep

12

2012

Why we should lie about the weather (and maybe more)

Nate Silver (who else?) has written a great piece on weather prediction — “The Weatherman is Not a Moron” (NYT) — that covers both the proliferation of data in weather forecasting, and why the quantity of data alone isn’t enough. What intrigued me though was a section at the end about how to communicate the inevitable uncertainty in forecasts:

…Unfortunately, this cautious message can be undercut by private-sector forecasters. Catering to the demands of viewers can mean intentionally running the risk of making forecasts less accurate. For many years, the Weather Channel avoided forecasting an exact 50 percent chance of rain, which might seem wishy-washy to consumers. Instead, it rounded up to 60 or down to 40. In what may be the worst-kept secret in the business, numerous commercial weather forecasts are also biased toward forecasting more precipitation than will actually occur. (In the business, this is known as the wet bias.) For years, when the Weather Channel said there was a 20 percent chance of rain, it actually rained only about 5 percent of the time.

People don’t mind when a forecaster predicts rain and it turns out to be a nice day. But if it rains when it isn’t supposed to, they curse the weatherman for ruining their picnic. “If the forecast was objective, if it has zero bias in precipitation,” Bruce Rose, a former vice president for the Weather Channel, said, “we’d probably be in trouble.”

My thought when reading this was that there are actually two different reasons why you might want to systematically adjust reported percentages ((ie, fib a bit) when trying to communicate the likelihood of bad weather.

But first, an aside on what public health folks typically talk about when they talk about communicating uncertainty: I’ve heard a lot (in classes, in blogs, and in Bad Science, for example) about reporting absolute risks rather than relative risks, and about avoiding other ways of communicating risks that generally mislead. What people don’t usually discuss is whether the point estimates themselves should ever be adjusted; rather, we concentrate on how to best communicate whatever the actual values are.

Now, back to weather. The first reason you might want to adjust the reported probability of rain is that people are rain averse: they care more strongly about getting rained on when it wasn’t predicted than vice versa. It may be perfectly reasonable for people to feel this way, and so why not cater to their desires? This is the reason described in the excerpt from Silver’s article above.

Another way to describe this bias is that most people would prefer to minimize Type II Error (false negatives) at the expense of having more Type I error (false positives), at least when it comes to rain. Obviously you could take this too far — reporting rain every single day would completely eliminate Type II error, but it would also make forecasts worthless. Likewise, with big events like hurricanes the costs of Type I errors (wholesale evacuations, cancelled conventions, etc) become much greater, so this adjustment would be more problematic as the cost of false positives increases. But generally speaking, the so-called “wet bias” of adjusting all rain prediction probabilities upwards might be a good way to increase the general satisfaction of a rain-averse general public.

The second reason one might want to adjust the reported probability of rain — or some other event — is that people are generally bad at understanding probabilities. Luckily though, people tend to be bad about estimating probabilities in surprisingly systematic ways! Kahneman’s excellent (if too long) book Thinking, Fast and Slow covers this at length. The best summary of these biases that I could find through a quick Google search was from Lee Merkhofer Consulting:

 Studies show that people make systematic errors when estimating how likely uncertain events are. As shown in [the graph below], likely outcomes (above 40%) are typically estimated to be less probable than they really are. And, outcomes that are quite unlikely are typically estimated to be more probable than they are. Furthermore, people often behave as if extremely unlikely, but still possible outcomes have no chance whatsoever of occurring.

The graph from that link is a helpful if somewhat stylized visualization of the same biases:

In other words, people think that likely events (in the 30-99% range) are less likely to occur than they are in reality, that unlike events (in the 1-30% range) are more likely to occur than they are in reality, and extremely unlikely events (very close to 0%) won’t happen at all.

My recollection is that these biases can be a bit different depending on whether the predicted event is bad (getting hit by lightning) or good (winning the lottery), and that the familiarity of the event also plays a role. Regardless, with something like weather, where most events are within the realm of lived experience and most of the probabilities lie within a reasonable range, the average bias could probably be measured pretty reliably.

So what do we do with this knowledge? Think about it this way: we want to increase the accuracy of communication, but there are two different points in the communications process where you can measure accuracy. You can care about how accurately the information is communicated from the source, or how well the information is received. If you care about the latter, and you know that people have systematic and thus predictable biases in perceiving the probability that something will happen, why not adjust the numbers you communicate so that the message — as received by the audience — is accurate?

Now, some made up numbers: Let’s say the real chance of rain is 60%, as predicted by the best computer models. You might adjust that up to 70% if that’s the reported risk that makes people perceive a 60% objective probability (again, see the graph above). You might then adjust that percentage up to 80% to account for rain aversion/wet bias.

Here I think it’s important to distinguish between technical and popular communication channels: if you’re sharing raw data about the weather or talking to a group of meteorologists or epidemiologists then you might take one approach, whereas another approach makes sense for communicating with a lay public. For folks who just tune in to the evening news to get tomorrow’s weather forecast, you want the message they receive to be as close to reality as possible. If you insist on reporting the ‘real’ numbers, you actually draw your audience further from understanding reality than if you fudged them a bit.

The major and obvious downside to this approach is that people know this is happening, it won’t work, or they’ll be mad that you lied — even though you were only lying to better communicate the truth! One possible way of getting around this is to describe the numbers as something other than percentages; using some made-up index that sounds enough like it to convince the layperson, while also being open to detailed examination by those who are interested.

For instance, we all the heat index and wind chill aren’t the same as temperature, but rather represent just how hot or cold the weather actually feels. Likewise, we could report some like “Rain Risk” or “Rain Risk Index” that accounts for known biases in risk perception and rain aversion. The weather man would report a Rain Risk of 80%, while the actual probability of rain is just 60%. This would give us more useful information for the recipients, while also maintaining technical honesty and some level of transparency.

I care a lot more about health than about the weather, but I think predicting rain is a useful device for talking about the same issues of probability perception in health for several reasons. First off, the probabilities in rain forecasting are much more within the realm of human experience than the rare probabilities that come up so often in epidemiology. Secondly, the ethical stakes feel a bit lower when writing about lying about the weather rather than, say, suggesting physicians should systematically mislead their patients, even if the crucial and ultimate aim of the adjustment is to better inform them.

I’m not saying we should walk back all the progress we’ve made in terms of letting patients and physicians make decisions together, rather than the latter withholding information and paternalistically making decisions for patients based on the physician’s preferences rather than the patient’s. (That would be silly in part because physicians share their patients’ biases.) The idea here is to come up with better measures of uncertainty — call it adjusted risk or risk indexes or weighted probabilities or whatever — that help us bypass humans’ systematic flaws in understanding uncertainty.

In short: maybe we should lie to better tell the truth. But be honest about it.

Sep

09

2012