Dan Drezner takes a look at John Kerry’s “new and improved!” misery index:

Every index can be challenged on the quality of the data that goes into it, and the weights that are assigned to the various components that make up the overall figure. A lack of transparency about methodology is also a valid criticism. For example, in my previous post on the competitiveness of different regions in the global information economy, the company responsible for the rankings provides little (free) information on how the index was computed. That’s a fair critique.

Even when the methodology is transparent, there can still be problems.

This is a subject near-and-dear to my heart. In quantitative social science, your econometric model is only as useful as your indicators; a crappy indicator renders the whole model essentially useless.

Unfortunately, our ways of dealing with the problem of how well an indicator reflects a concept leave a lot to be desired; “face validity”—which boils down to “I think the indicator reflects the concept, so we’ll a priori assume it does”—is relied on, even by good scholars, to an extent that will make you blanch. Even seemingly obvious indicators, like responses to survey questions, are often woefully inadequate for measuring “true” concepts (in the case of public opinion research, attitudes and predispositions).

Building an index helps with some of these problems—if your measurement error—but introduces others (like ascribing valid weights to the items, as Dan points out). A few cool tools, like factor analysis and its cousin principal components analysis, are designed to help in finding weights, but even they have problems and limitations, most of which basically boil down to the fact that human judgment is still involved in the process.

I’ve arrived safe and sound in Chi-town. Illinois has to be the most boring state in the nation; the whole state is flat as a pancake as soon as you get north of Mount Vernon (where I-64 crosses the state on its way between Louisville and St. Louis), and it doesn’t get more exciting until you can see the Sears Tower about 300 miles (500 km) later.

I even tried going a different way than usual (“Surely this lake on the map means topography nearby,” I thought erroneously), and all I got for my effort was an extra hour of staring at endless farmland, although I at least got off I-57—in other words, at least the flat, boring farmland was different flat, boring farmland. (From Effingham, I took Ill. 32 to Ill. 121 to Decatur, then took U.S. 51 north to Bloomington, then I-55 into Chicago.)

Speaking of Effingham, that giant cross is just freaky. Say what you will about Southern Baptists, but at least they have the good taste not to inflict something so immensely gaudy on the motoring public (preferring, instead, giant fields of tiny crosses or trinities of smaller crosses that aren’t hazards to unsuspecting amateur pilots).

Now, off to get me a light dinner and to finish getting junk out of my car.