Not having analyzed the data (a big caveat for a social scientist, mind you) I’ll agree with the critics who aren’t buying the evidence from a Heritage report that suggests that “abstinance pledge” programs work. Not that the story makes that much sense, since it’s clear the author doesn’t actually know anything about social scientific research and just relies on an expert and the authors of the original study to rebut the paper.

But Matthew Yglesias’ critique really goes off the rails. First he complains, “the study was not peer-reviewed, is unpublishable in real academic journals, uses an unreliable data source, and only supports the conclusion when you use a non-standard test for statistical significance.”

The first two critiques are bizarre, since (a) it has never been submitted for peer review and (b) we don’t know whether or not it’s publishable, since submission for peer review hasn’t happened yet; the lack of publishability is an opinion expressed by someone in the article, not a factual statement. They don’t use any “non-standard test”; they use a p-value of 0.10 as their cutoff, which isn’t the traditional 0.05 and not quite as convincing as 0.05, but isn’t inherently invalid either, and confidence levels aren’t tests (examples of tests are “t” tests and “Wald” tests; p values are the results of statistical tests).

The only critique that’s even vaguely valid is that the data source is unreliable, as it relies on self-reporting by respondents of their behavior. This is a problem, to the extent you believe that people who have signed abstinence pledges are more likely to lie about their sexual activity than those who haven’t. I’ll concede that it’s possible that that’s the case. Mind you, Heritage didn’t come up with the data—HHS did—and trying to get people to accurately self-report anything is harder than it looks.

Then Yglesias turns and goes completely bizarro:

The only newsworthy information in the story is that the Bush Department of Health and Human Services has decided for some reason to start contracting out research on controversial questions to an ideological think tank that is non-partisan in name only, rather than to proper independent analysts.

There is no evidence in the story that Heritage was working under any sort of HHS contract. On the contrary, Heritage appears to have analyzed data, produced under HHS and CDC contract, which is in the public domain.* They then presented their results at a government-sponsored conference. The next step would be to fix any problems in the paper (and the article suggests there were some), and then submit the paper to a peer-reviewed journal. That’s how social science is done.

Now, mind you, it might be premature for the New York Times to be calling attention to this story, but given public interest in the issue—and the Times’ possible interest in discrediting this evidence, not that I’d suspect the paper of having an ideological bias in its reporting decisions—I’m not sure I can fault them for covering preliminary results that (potentially) rebut a serious critique of administration policy.

* If the CDC had helped fund either analysis, it would be traditional for the studies to acknowledge the funding at the beginning of the paper in a note. I think it’s more likely that the Times meant to say that the CDC helped fund the HHS survey, not the Heritage study.

Michele, posting basically the same thing I wrote in comments to this post by someone in the “non-reciprocal linking” school (exponded at length here), except it disappeared into the ether:

I don’t really care what YOUR etiquette/rules for trackbacks are. I know what MY rule is: Don’t trackback to my posts unless you’ve linked/referenced them. I see no reason for someone to go through the trouble of sending a trackback which basically says “hey, I’ve written about the same thing as you, but I didn’t reference your post on it at all. However, I’m going to use this nifty automated feature to leave a URL to MY post on your blog!” That’s just god damn RUDE.

I also HATE when trackback is used as a feature to say “I wrote about a subject that you seem to care about and instead of emailing you a note to say hey, check out this post, I think you’ll be interested in it, I’ll just lazily send off a trackback to a completely UNRELATED post of yours, most likely your most recent post, thereby informing you that I’ve said something I consider important and leaving a URL to my very important post in your completey UNRELATED post!”

See where I’m going with this? Stop doing it. It’s arrogant. Take two freaking minutes to send an email. Or don’t trackback at all.

I will delete any trackbacks that don’t reference the post they track. Don’t be an such a self-absorbed ass all your life, ok? Show some manners.

That’s my policy too… if you bothered to find the trackback URL, you’ve already found out how to link to my post. Or, to put it another way, if you want me to send you some of my readers, the least you can do is send me some of yours too. Frankly, to not do so isn’t just rude—it’s spamming, and just because you’re not selling something doesn’t make it not spam.

I feel like I actually achieved something this morning—I finished packing all the books in my office, except the ones I’m using for classes this summer (and the ones I’m bequeathing to my successor):

Now I get to deal with the shelves full of books at home.