Archive for April, 2012

Group vs. individual uses of data

Andrew Gelman notes that, on the subject of value-added assessments of teachers, “a skeptical consensus seems to have arisen…” How did we get here?

Value-added assessments grew out of the push for more emphasis on measuring success through standardized tests in education — simply looking at test scores isn’t OK because some teachers are teaching in better schools or are teaching better-prepared students. The solution was to look at how teachers’ students improve in comparison to other teachers’ students. Wikipedia has a fairly good summary here.

Back in February New York City released (over the opposition of teachers’ unions) the value-added scores of some 18,000 teachers. Here’s coverage from the Times on the release and reactions.

Gary Rubinstein, an education blogger, has done some analysis of the data contained in the reports and published five posts so far: part 1, part 2, part 3, part 4, and part 5. He writes:

For sure the ‘reformers’ have won a battle and have unfairly humiliated thousands of teachers who got inaccurate poor ratings. But I am optimistic that this will be be looked at as one of the turning points in this fight. Up until now, independent researchers like me were unable to support all our claims about how crude a tool value-added metrics still are, though they have been around for nearly 20 years. But with the release of the data, I have been able to test many of my suspicions about value-added.

I suggest reading his analysis in full, or at least the first two parts.

For me one early take-away from this — building off comments from Gelman and others — is that an assessment might be a useful tool for improving education quality overall, while simultaneously being a very poor metric for individual performance. When you’re looking at 18,000 teachers you might be able to learn what factors lead to test score improvement on average, and use that information to improve policies for teacher education, recruitment, training, and retention. But that doesn’t mean one can necessarily use the same data to make high-stakes decisions about individual teachers.

What happened?

What happened during the 2007-8 financial crisis? Here’s a reading from my classes that I think may be of interest to a broader audience: “Getting up to Speed on the Financial Crisis: A One-Weekend-Reader’s Guide” by Gary B. Gorton and Andrew Metrick, writing in January 2012 (PDF from NBER).

Covering 16 sources (academic papers, a few reports by institutions, and Congressional testimony by Bernanke) Gorton and Metrick provide a timeline of the crisis, some historical perspective on past banking crises, the build-up to this crisis, phases of the crisis itself, and government responses.

It’s just 34 pages and interesting throughout — the only shortcoming is that the PDF is rendered in Calibri.

A related article is Andrew Lo’s “Reading About the Financial Crisis: A 21-Book Review” (PDF), which includes this:

No single narrative emerges from this broad and often contradictory collection of interpretations, but the sheer variety of conclusions is informative, and underscores the desperate need for the economics profession to establish a single set of facts from which more accurate inferences and narratives can be constructed.

Discussions of causes are difficult when you don’t agree on the simpler matters of what actually happened — which speaks to the importance of trying to simply get at (as Gorton and Metrick are trying to do) an account of what happened.

23

04 2012

On food deserts

Gina Kolata, writing for the New York Times, has sparked some debate with this article: “Studies Question the Pairing of Food Deserts and Obesity”. In general I often wish that science reporting focused more on how the new studies fit in with the old, rather than just the (exciting) new ones. On first reading I noticed that one study is described as having explored the association of “the type of food within a mile and a half of their homes” with what people eat.

This raised a little question mark in my mind, as I know that prior studies have often looked at distances much shorter than 1.5 miles, but it was mostly a vague hesitation. And if you didn’t know that before reading the article, then you’ve missed a major difference between the old and new results (and one that could have been easily explained). Also, describing something as “an article of faith when it’s arguably something more like “the broad conclusion draw from most most prior research“… that certainly established an editorial tone from the beginning.

Intrigued, I sent the piece to a friend (and former public health classmate) who has work on food deserts, to get a more informed reaction. I’m sharing her thoughts here (with permission) because this is an area of research that I don’t follow as closely, and her reactions helped me to situate this story in the broader literature:

1. This quote from the article is so good!

“It is always easy to advocate for more grocery stores,” said Kelly D. Brownell, director of Yale University’s Rudd Center for Food Policy and Obesity, who was not involved in the studies. “But if you are looking for what you hope will change obesity, healthy food access is probably just wishful thinking.”

The “unhealthy food environment” has a much bigger impact on diet than the “healthy food environment”, but it’s politically more viable to work from an advocacy standpoint than a regulatory standpoint. (On that point, you still have to worry about what food is available – you can’t just take out small businesses in impoverished neighborhoods and not replace it with anything.)

2. The article is too eager to dismiss the health-food access relationship. There’s good research out there, but there’s constant difficulty with tightening methods/definitions and deciding what to control for. The thing that I think is really powerful about the “food desert” discourse is that it opens doors to talk about race, poverty, community, culture, and more. At the end of the day, grocery stores are good for low-income areas because they bring in money and raise property values. If the literature isn’t perfect on health effects, I’m still willing to advocate for them.

3. I want to know more about the geography of the study that found that low-income areas had more grocery stores than high-income areas. Were they a mix of urban, peri-urban, and rural areas? Because that’s a whole other bear. (Non-shocker shocker: rural areas have food deserts… rural poverty is still a problem!)

4. The article does a good job of pointing to how difficult it is to study this. Hopkins (and the Baltimore Food Czar) are doing some work with healthy food access scores for neighborhoods. This would take into account how many healthy food options there are (supermarkets, farmers’ markets, arabers, tiendas) and how many unhealthy food options there are (fast food, carry out, corner stores).

5. The studies they cite are with kids, but the relationship between food insecurity (which is different, but related to food access) and obesity is only well-established among women. (This, itself, is not talked about enough.) The thinking is that kids are often “shielded” from the effects of food insecurity by their mothers, who eat a yo-yo diet depending on the amount of food in the house.

My friend also suggested the following articles for additional reading:

More on microfoundations

Last month I wrote a long-ish post describing the history of the “microfounded” approaches to macroeconomics. For a while I was updating that post with links to recent blog posts as the debate continued, but I stopped after the list grew too long.

Now Simon Wren-Lewis has written two more posts that I think are worth highlighting because they come from someone who is generally supportive of the microfoundations approach (I’ve found his defense of the general approach quite helpful), but who still has some specific critiques. The end of his latest post puts these critiques in context:

One way of reading these two posts is a way of exploring Krugman’s Mistaking Beauty for Truth essay. I know the reactions of colleagues, and bloggers, to this piece have been quite extreme: some endorsing it totally, while others taking strong exception to its perceived targets. My own reaction is very similar to Karl Smith here. I regard what has happened as a result of the scramble for austerity in 2010 to be in part a failure of academic macroeconomics. It would be easy to suggest that this was only the result of unfortunate technical errors, or political interference, and that otherwise the way we do macro is basically fine. I think Krugman was right to suggest otherwise. Given the conservative tendency in any group, an essay that said maybe there might just be an underlying problem here would have been ignored. The discipline needed a wake-up call from someone with authority who knew what they were talking about. Identifying exactly what those problems are, and what to do about them, seems to me an important endeavour that has only just begun.

Here are his two posts:

  1. The street light problem: “I do think microfoundations methodology is progressive. The concern is that, as a project, it may tend to progress in directions of least resistance rather than in the areas that really matter – until perhaps a crisis occurs.”
  2. Ideological bias: “In RBC [Real Business Cycle] models, all changes in unemployment are voluntary. If unemployment is rising, it is because more workers are choosing leisure rather than work. As a result, high unemployment in a recession is not a problem at all…. If anyone is reading this who is not familiar with macroeconomics, you might guess that this rather counterintuitive theory is some very marginal and long forgotten macroeconomic idea. You would be very wrong.”

22

04 2012

Name that quote

I’m reading Evolving Economics, a highly-regarded history of economic thought by Agnar Sandmo. I thought one tidbit early on was quite interesting: it comes in the course of a discussion of a once-common method of charging tolls based on the weight of carriages. Sandmo quotes an economist who recommended different rates for luxury versus other transport.

Thus, “…the indolence and vanity of the rich is made to contribute in a very easy manner to the relief of the poor, by rendering cheaper the transportation of heavy goods to all the different parts of the country.”

Who said that? Answer below the fold…

Read the rest of this entry →

20

04 2012

Walrasian equilibrium

For econ nerds only:

Here’s Wikipedia on general equilibrium and Léon Walras. Wolfram Alpha also has a great little demonstration of Walrasian equilibrium where you can play around with the parameters.

Posted with the permission of my classmate and econ comic artist Marian Messing. Prior econ humor here and here.

17

04 2012

Monday Miscellany

  • Today is the 2012 “Day Without Dignity.” What’s that? Saundra S of the blog Good Intentions Are Not Enough explains: “A Day Without Dignity was started last year as a counter-campaign to TOMs Shoes One Day Without Shoes event. With so many Whites in Shining Armor projects making the news we decided this year to focus on local champions instead.” There are 24 posts and counting for Local Champions linked here, or you can follow #localchampions on Twitter.
  • Lee Crawfurd shares this post by Gabriel Demombynes at the World Bank’s Development Impact blog, a fascinating comparison of an intervention implemented simultaneously by an NGO and a government, resulting in quite different results.
  • Alex Evans explores what comes after the Millennium Development Goals.
  • Do child sponsorship programs actually work? Maybe so.
  • “Interactive Islands of Mankind” is a simple but sweet interactive tool by Derek Watkins that lets you see variations in population density around the world.
  • Finally, a hilarious Public Service Announcement regarding safe sex for senior citizens is going viral (pun intended). No comment on whether this will be effective, but it is certainly getting some attention.

16

04 2012

Non-representative sample of Hunger Games responses

I had the idea for the Hunger Games survival analysis post Tuesday afternoon and published it about 24 hours later (and yes, in the meantime I did sleep, eat, and do a bit of real work as well). I thought it might hit a nerdy nerve by meshing pop culture and stats, and I was right. Three days later it’s been read by over 12,000 people on my site alone, and the average time on page is long enough that I think folks are actually reading it and not just looking at the pretty pictures. It was picked up by Andrew Gelman and Jezebel (a Venn diagram with only this in the middle, I bet) and everyone from Stata to Discover Magazine shared it on Twitter.

All that to say, I think there’s a market for explaining statistics and concepts from social science (I tried to work in some political science, economics, and psychology research) using pop culture tie-ins, so I may do some more of this.

For now I want to share some of the humorous reactions I’ve seen:

  • A classmates who is familiar with survival analysis but hasn’t read the books saw the graphs and her immediate response was “Oh no, what happened on the first day? Those poor children!”
  • Richard Williams, commenting on the Stata listserv discussion: “If Stata can win over the Hunger Games crowd, SAS & SPSS are finished.”
  • One of the comments on Metafilter kind of misses the point: “I love statistics, but come on: The major finding here is that Suzanne Collins did a good job creating a fictional dataset that shows some significant differences between groups. Yes, that’s because statistics measures deviations from randomness, and Collins *made up the data* as part of her novel’s plot.” Shocking.
  • A friend to a friend of mine on Gchat: “he used Stata for a Good Thing. It was Interesting. That’s im-[ahem]-Possible and he Did It.
  • Finally, the Teaching Assistant from last semester’s generalized linear models class threatened to grade it as an assignment. Next time I’ll use data where the assumption of the model (proportional hazards) aren’t clearly violated…

14

04 2012

An application of survival analysis to the Hunger Games (seriously)

I just finished what is quite possibly the nerdiest thing I’ve ever written:  “Hunger Games survival analysis.” I manage to pull in articles from Matt Yglesias and Erik Kain and discuss tesserae inflation, Prospect Theory, demographics, research by Acemoglu and Robinson and by Michael Clemens, game theory, coordination failures, arguments for open data, and of course the namesake survival analysis. Complete with Kaplan-Meier survival estimator graphs and all:

I posted it as a page rather than a blog post to make some of the formatting easier, so please click through to read the real thing.

11

04 2012

Outbreak control

The latest MMWR (Morbidity and Mortality Weekly Report) from the CDC  has a summary of a meningitis outbreak in Oklahoma and how public health authorities responded: “Outbreak of Meningococcal Disease Associated with an Elementary School – Oklahoma, March 2010.”

MMWR reports have a consistent style that I think is helpful for this sort of notice: they’re short with tight editing and little superfluous information. They also often present harrying situations that are made more disturbing by the clinical detachment. In this case:

Five cases of meningococcal disease (including one probable case) were identified among four elementary school students and one high school student. Two students died; two recovered fully, and one survivor required amputation of all four limbs and facial reconstruction.

They also often include a helpful summary answering three questions: 1) What is already known on this topic? 2) What is added by this report? and 3) What are the implications for public health practice? I think some other publications (especially in the social sciences) would benefit from this helpful little formatting addition.

05

04 2012