Saturday, 1 November 2003

More on selection bias

Glenn Reynolds links to a Lynxx Pherrett dissection of the alleged ‘pay-for-play’ nature of Iraqi reconstruction contracts. The key graf, I think, is:

How do I know CPI is dealing from a stacked-deck? As Marshall Brodien said, "It's easy, once you know the secret!" CPI only looked at companies that were awarded contracts, then examined the companies' political contribution history and any connections to current or former government officials. What CPI never looked at, and according to their methodology never attempted to look at, was the political contribution and governmental connection histories of the losing submitters. In other words, there is nothing against which their results can be compared. Businesses make political contributions — we know that. People leave government service and go to work in the private sector — we know that. Thus, no matter what major company wins a contract, it is likely that they have 1) made political donations in the past—CPI researched contributions all the way back to 1990—and 2) employ some former government officials. Unless CPI can show that the contract winners made larger political contributions and employed more or higher-level ex-government officials, their report cannot support Lewis' charge of "a stench of political favoritism and cronyism."

In other words, CPI selected on the dependent variable. Quality social science here, folks…

Dan Drezner has more.