The American Spectator

And so on, ad infinitum, for every variable that can be thought of—and many that can’t. Anyone attempting to answer a question as simple as “does the name Fast- Mart increase sales” faces such an overwhelming density of possible causal factors that it is impossible for him to structure the study without making assumptions about which variables to include and which to exclude.

In other words, there is simply no way to answer the question without the risk of leaving out an important variable. He could present a plausible answer to the question of whether the name “Fast- Mart” increases sales, but that answer would inevitably reflect his own presuppositions.

Manzi concludes that RFTs present a possible resolution to some of our endless public debates, like the one over the stimulus. In part, he believes this because they have already made an impact in business. In particular, Manzi points to the success of Capital One, a credit card company that grew rapidly throughout the ’90s using RFTs to establish marketing strategies. The company would test new advertisements by randomly selecting treatment and control households, and then use the results to establish larger campaigns. Capital One ran about 60,000 such tests in 2000, according to Manzi, and has quickly grown into the Fortune 500 company it is today. Google, notably, also has incorporated RFTs into its business strategy, running 12,000 in 2009.

There are a few examples of RFTs being done for governments. The 1996 welfare reform, for one example, was partly motivated by a series of statelevel experiments. Manzi believes that far more could be done, in health care, crime prevention, education, and a host of other areas. The federal government, in particular, should be performing thousands of experiments a year.

A NUMBER of social scientists have begun to bring RFTs to the forefront of their specializations. The MIT economists Abhijit Banerjee and Esther Duflo use field experiments to study global poverty, by testing small-scale interventions such as distributing mosquito nets. Duflo won the John Bates Clark Medal, given to the best economist under age 40, in 2010. Manzi also cites the education economist Roland Fryer and the Chicago economist John List as RFT pioneers. If RFTs gain prominence in academia, they can also enter use in policymaking.

Of course, RFTs suffer from their own shortcomings. Mosquito nets that save lives in Africa may be useless in India, and school vouchers that increase test scores in Milwaukee could easily flop in Baltimore. Furthermore, RFTs are more likely to yield modest, specific answers to narrowly defined questions, rather than point to sweeping conclusions.

Manzi proposes that, ideally, many different functions of the federal government should be spun off to the states and then tested rigorously using RFTs as the states experiment with different approaches. In essence, he suggests that the government engage in the same sort of trial-and-error process that defines the private sector.

One obvious objection to Manzi’s enthusiasm for RFTs is that his technocratic view of their possibilities for government is divorced from reality. Democratic politics, after all, is about competing interests and coalitions, not best practices. Furthermore, it’s unlikely that governors or mayors, let alone legislators, are interested in running experiments that could prove to be failures.

Nevertheless, even if their use in government is generations away, RFTs could, in the meantime, dispel many myths that have entered the mainstream through suspect research. Uncontrolled is worthwhile for its reminder that most studies that shape public debates report little more than the authors’ biases, and have no predictive power whatsoever. In other words, Uncontrolled is a call for humility about what we know—and a little of that goes a long way.