Econometric Modeling as Junk Science

Ehrlich was not persuaded by these critics, and found flaws in their work. He remains a lonely true believer in the validity of his model. In a recent interview (Bonner and Fessendren, 2000) he insisted that "if variations like unemployment, income inequality, likelihood of apprehension and willingness to use the death penalty are accounted for, the death penalty shows a significant deterring effect."

Myth Four: Legalized Abortion Caused the Crime Drop in the 1990s.

In 1999, John Donohue and Steven Levitt released a study with a novel explanation of the sharp decline in murder rates in the 1990s. They argued that the legalization of abortion by the U.S. Supreme Court in 1973 caused a decrease in the birth of unwanted children, a disproportionate number of whom would have grown up to be criminals. The problem with this is that the legalization of abortion was a one-time historical event and there are too little data for a valid regression analysis. The results are likely to vary depending on how data are selected for analysis. In this case, as James Fox (2000: 303) pointed out: "by employing a single statistic summarizing change over this twelve-year span, [Donohue and Levitt] miss most of the shifts in crime during this period - the upward trend during the late 1980s crack era and the downward correction in the post-crack years. This is something like studying the effects of moon phases on ocean tides but only recording data for periods of low tide."

When I was writing this article, I included a sentence stating "soon another regression analyst will probably reanalyze the same data and reach different conclusions." A few days later, my wife handed me a newspaper story about just such a study. The author was none other than John Lott of Yale, together with John Whitley of the University of Adelaide. They crunched the same numbers and concluded that "legalizing abortion increased murder rates by around about 0.5 to 7 percent" (Lott and Whitely, 2001).

Why such markedly different results? Each set of authors simply selected a different way to analyze an inadequate body of data. Econometrics cannot make a valid general law out of the historical fact that abortion was legalized in the 1970s and crime went down in the 1990s. We would need at least a few dozen such historical experiences for a meaningful statistical test.

Myth Five: Welfare Reform Will Throw a Million Children into Poverty.

On August 1, 1996, as the United States Senate considered an epochal change in welfare policies, the Urban Institute issued a widely publicized report claiming to demonstrate that: the proposed welfare reform changes would increase poverty and reduce incomes of families in the lowest income group... We estimate that 2.6 million more persons would fall below the poverty line as a result, including 1.1 million children. (Urban Institute, 1996, p. 1)

Welfare advocates rallied around this prediction, but policy makers were not persuaded. Senators who supported the reform simply did not believe that social scientists could make valid predictions of that sort. And they were right. The Urban Institute could not even predict the direction of change, let alone its magnitude. Child poverty went down, not up, after the welfare reform.

The Urban Institute's model was much more complex than the other models we have examined in this paper, but the added complexity seems only to have compounded the problem. Using sophisticated "microsimulation" techniques, they took correlations that existed in the past, fed them into complex equations, then treated these equations as general laws. All their mathematics was based on the assumption that nothing fundamental would change, in which case, of course, welfare reform would fail. All the model did was produce numbers to illustrate their arguments and make them appear scientific. But the point of the reform was to change things, and it did.

Why Regression Fails