Share |

Do Negative Test Results Mean School Choice Has Failed? Perhaps the Opposite!

Neal McCluskey

It is understandably considered bad news for school choice when
a study comes out finding negative effects on test scores,
especially one using a “gold standard,” randomized control trial
design. But context is crucial for understanding such findings, and
this may be especially true for a new study of vouchers
in Washington, D.C.

The researchers studied various impacts one year after families
applied to the District of Columbia Opportunity Scholarship
Program. Studied students participated in a lottery for vouchers,
which is key because who wins or loses is random, automatically
controlling for variables that can powerfully affect outcomes, such
as family income or motivation levels.

nd those who won and used the voucher to attend a private school
— what policymakers likely care most about — saw
slightly lower scores than non-users. Reading scores were also
slightly lower, but were not statistically significant, meaning the
researchers could not be confident the differences were other than
the result of random chance.

So what’s the good news here? Surely no one can be pleased that
choice appears to lead to lower math scores.

For one thing, on other measures the program fared better.
Parent and student satisfaction with their schools were higher for
both lottery winners and winners who used their vouchers, though
the results did not reach statistical significance. Both winners
and users were also more likely to perceive their schools as safe,
though statistical significance was only reached for parents. And
for kids in sixth through 12th grade, the program had a positive
impact on parents’ involvement in education-related activities at

Then there’s this: the
sum of education is far more than standardized test

Much more important, the test results may well be the result of
choice working, not failing, in Washington. You see,
families in Washington have lots of choices.

First, Washington is a city, so people who live there can put
pressure on the district by saying, “Shape up or we’ll move to
Virginia or Maryland.” More directly, Washington has a huge charter
school sector. Indeed, 42 percent of the study’s control group
attended charter schools, and 10 percent attended private schools
despite losing the lottery. Only 48 percent attended traditional
public schools.

Quite possibly because of so much choice pressure — most
imposed by Congress — the public schools in Washington have
seen marked increases in achievement. According to the National
Assessment of Educational Progress, since the mid-1990s
achievement in Washington
has risen at a rate appreciably
outpacing the national average. Charter schooling in Washington
started in 1996, and the voucher program was created in 2004.

Alas, charter schools — tuition-free public schools that
in many ways seem private — have likely
hurt Washington’s private sector
, which was already struggling
against free traditional public schools and decades of changes in
the Roman Catholic Church, whose schools predominate in the private
sector. If nothing else, struggling to survive cannot be positive
for staff morale.

So no one should be surprised that a voucher program enrolling
fewer than 1,200 students, which has been
repeatedly threatened with extinction, does not have
powerful testing effects. Oh, and a maximum voucher is roughly
$8,000 for grades K-8, and $13,000 for 9-12.
Meanwhile, the traditional public schools spend a whopping $30,000 per-pupil, and charters get about

Given the gaping funding disparities, it may seem amazing that
“gold-standard” research
found that Washington voucher students
performed on par with the control group on tests, and beat it
soundly on high school graduation rates. And in the current report,
the math difference was only about 7 percentile points between
voucher users and the control group-appreciable, but not

Then there’s this: the sum of education is far more than
standardized test scores. Indeed, the nation has seen a backlash against education reduced to such
narrow measures, which
may not predict future success
. And it may be that people want
things out of schooling that simply cannot be easily tested,
ranging from safety, to strong moral values, to appreciation for
the arts, to just a sense of fulfillment. Which is why it may be a
good sign that even if it adversely affected test scores, schools
chosen by lottery winners spent less time on math and reading
instruction than control group schools. They may instead have been
devoting time to music, or field trips, or myriad other very valuable

Are negative testing impacts good? All other things equal, no.
But all other things are not equal, especially in Washington.

Neal McCluskey is a contributor to the Washington Examiner’s Beltway Confidential blog. He is the director of the Cato Institute’s Center for Educational Freedom and maintains Cato’s Public Schooling Battle Map.