Wednesday, 4 August 2004

Once more into the breach

Stephen Bainbridge (via Glenn Reynolds) isn’t impressed with the use of NOMINATE scores to cast John Kerry as more of a centrist; nor is he particularly thrilled with methods like NOMINATE to begin with:

Personally, I find the interest group scores much more accessible and transparent. For one thing, NOMINATE counts all nonunanimous roll calls, which can include a lot of procedural and uncontroversial (even nonpartisan) bills. The interest group rankings focus on bills that really tell us something about the political philosophy of the candidate in question. For another, the interest group ratings are widely used both by the media and, perhaps more important, by politicians themselves.

I’d respond that NOMINATE (and related methods) are preferable to interest group scores precisely because they count all nonunanimous roll calls; this avoids the selection effect where interest groups choose, say, twenty “key” votes as a litmus test for an entire session. And, presumably, those who vote on party lines on “nonpartisan” and “uncontroversial” bills are even more partisan than those who join with their natural opposition. Another worthy point in favor of NOMINATE: the “procedural” versus “substantive” distinction is largely subjective; cloture votes in the Senate, for example, are technically procedural motions to end debate (and potentially stop a filibuster), while procedural votes on rules in the House often have serious substantive consequences (by ruling certain amendments out-of-order, framing and controlling debate, and sometimes even amending the legislation in question).

Now, Clinton, Jackman, and Rivers are quite correct to point out that the statistical properties of NOMINATE are, at best, nebulous, although Lewis and Poole recently made a worthy effort to gain additional leverage on the bias and uncertainty of NOMINATE in Political Analysis. And, while some of the differences in the results of the techniques are the result of differences between the distributional assumptions of NOMINATE and the CJR scaling method* (which explains the differing positions of Kerry in years in which he missed a lot of roll calls), there are some good reasons to prefer the CJR technique—most notably, it’s significantly more tractable; you can estimate the model almost trivially using MCMCpack.

Anyway, for those with a morbid curiosity about these techniques, the latest American Political Science Review has an article by Clinton, Jackman, and Rivers called “The Statistical Analysis of Roll-Call Data,” which I recommend highly (and which you may or may not be able to access via this link).

I’ve posted previously on NOMINATE and related methods (again, in relation to John Kerry’s voting record) here and here. This is my entry in today’s Beltway Traffic Jam.

* Technically speaking, it’s a Normal-theory two-parameter item-response theory model in two dimensions. If that doesn’t give you a headache, consider reading Ordinal Data Modeling by Johnson and Albert, who exposit the one-dimensional case rather nicely in a later chapter of their text.

2 comments:

Any views expressed in these comments are solely those of their authors; they do not reflect the views of the authors of Signifying Nothing, unless attributed to one of us.

I’d tend to agree when talking about a single interest group. But if you sample multiple interest groups from both sides of the equation, and they all come up with the same result, isn’t that more interesting?

After all, it’s not like “liberal” or “conservative” are terms with precise meanings. Letting people who embrace those labels define them, and measure performance against their definition, sounds to me like it introduces less bias (as long as several groups from both sides are included) than a single “unbiased” report, even if it does use proper methods.

 
[Permalink] 2. ralph phelan wrote @ Thu, 5 Aug 2004, 1:34 pm CDT:

“this avoids the selection effect where interest groups choose, say, twenty “key” votes as a litmus test for an entire session. ”

I prefer the interest-group rating because I assume the innterest groups know what they’re doing. It is in the organizational self-interest of a PFAW or NARAL to do a good job of identifying those who advance their interests and how well they do so. So I’m willing to take their word for it when they select certain votes as being, in their opinion, key.

“those who vote on party lines on “nonpartisan” and “uncontroversial” bills are even more partisan than those who join with their natural opposition”

What exactly do we mean by “partisan?”

It may well be that “nominate” methods are better at measuring “who is the most loyal Democrat.” But that is not necessarily the same as measuring “who is the most liberal.” The most liberal members may well stray from the party line on issues where the party leadership has decided to compromise their principles.

 
Comments are now closed on this post.