On regressions and thinking

Thesis: thinking quantitatively changes the way we frame and answer questions in ways we often don’t notice.

James Robinson, of Acemoglu and Robinson fame (ie, Why Nations Fail@whynationsfailColonial Origins; Reversal of Fortune, and so forth), gave a talk at Princeton last week. It was a good talk, mostly about Why Nations Fail. My main thought during his presentation was that it’s simply very difficult to develop a parsimonious theory that covers something as complicated as the long-term economic and political development of the entire world! As Robinson said (quoting someone else), in social science you can say “more and more about less and less, or less and less about more and more.”

The talk was followed by some great discussion where several of the tougher questions came from sociologists and political economists. I think it’s safe to say that a lot of the skepticism of the Why Nations Fail thesis is centered around the beef that East Asian economics, and especially China, don’t fit neatly into it. A&R argue here on their blog — not to mention in their book, which I’ve alas only had time to skim — that China is not an exception to their theory, but I think that impression is still fairly widespread.

But my point isn’t about the extent to which China fits into the theory (that’s another debate altogether); it’s about what it means if or when China doesn’t fit into the theory. Is that a major failure or a minor one?  I think different answers to that question are ultimately rooted in a difference of methodological cultures in the social science world.

As social science becomes more quantitative, our default method for thinking about a subject can shift, and we might not even notice that it’s happening. For example, if your main form of evidence for a theory is a series of cross-country regressions, then you automatically start to think of countries as the unit of analysis, and, importantly, as being more or less equally weighted. There are natural and arguably inevitable reasons why this will be the case: states are the clearest politicoeconomic units, and even if they weren’t they’re simply the unit for which we have the most data. While you might (and almost certainly should!) weight your data points by population if you were looking at something like health or individual wealth or well-being, it makes less sense when you’re talking about country-level phenomena like economic growth rates. So you end up seeing a lot of arguments made with scatterplots of countries and fitted lines — and you start to think that way intuitively.

When we switch back to narrative forms of thinking, this is less true: I think we all agree that all things being equal a theory that explains everything except Mauritius is better than a theory that explains everything except China. But it’s a lot harder to think intuitively about these things when you have a bunch of variables in play at the same time, which is one reason why multiple regressions are so useful. And between the extremes of weighting all countries equally and weighting them by population are a lot of potentially more reasonable ways of balancing the two concerns, that unfortunately would involve a lot of arbitrary decisions regarding weighting…

This is a thought I’ve been stewing on for a while, and it’s reinforced whenever I hear the language of quantitative analysis working its way into qualitative discussions — for instance, Robinson said at one point that “all that is in the error term,” when he wasn’t actually talking about a regression. I do this sort of thing too, and don’t think there’s anything necessarily wrong with it — until there is.  When questioned on China, Robinson answered briefly and then transitioned to talking about the Philippines, rather than just concentrating on China. If the theory doesn’t explain China (at least to the satisfaction of many), a nation of 1.3 billion, then explaining a country of 90 million is less impressive. How impressive you find an argument depends in part on the importance you ascribe to the outliers, and that depends in part on whether you were trained in the narrative way of thinking, where huge countries are hugely important, or the regression way of thinking, where all countries are equally important units of analysis.

[The first half of my last semester of school is shaping up to be much busier than expected — my course schedule is severely front-loaded — so blogging has been intermittent. Thus I’ll try and do more quick posts like this rather than waiting for the time to flesh out an idea more fully.]

25

02 2013

2 Comments Add Yours ↓

The upper is the most recent comment

  1. 1

    All true and valid, though the flip-side is that thinking qualitatively (or narratively) also affects the way we think in unexpected ways.

    For examples see Tyler Cowen on thinking in stories, Chimamanda Adichie on the danger of a single story, Reid Hastie on the psychology of simple narratives, or even Ken Burns (who says “all story is manipulation”).

  2. 2

    Yes yes! If anything I think the default is thinking in terms of stories and narratives and that being trained to think more quantitatively is the more necessary correction. Overall I think that thinking in stories is the root of more trouble – in part because it’s the default so it’s just the root of so much more thinking – but that there are worthwhile cautions for each approach. It’s also interesting that, when going back to the narrative side (ie, presenting results in a paper or book) more narrative space is given to examples or exceptions that would be particularly challenging.



Your Comment to Jacob AG