For those who haven't been following along at home, AFTie John's main beef is the research I've quoted with respect to the D.C. choice program:

On your decision to cite another study in this follow-up, you're just moving the goalposts.  If I were to go through it and find it doesn't quite say what you're telling me it says, would you just move on to another study? 

First off, I addressed his original complaint by quoting the key paragraph in its entirety rather than a few select sentences.  A corrupted sample might (and I do stress might) make the study problematic, but that hardly invalidates the whole thing.  (It's worth pointing out that even with sampling issues, the study still reflects improved scores from African-American students--just not as improved as in Dayton and NYC.)  Furthermore, I've always understood quoting further research as bolstering one's case. 

By the way, when I clicked on your link to the new study, it didn't take me to a study.  It took me to an article in Education Next (not a peer-reviewed journal) about a study supposedly located at www.edmattersmore.org.  When I click on that URL, there's nothing there. 

Sorry about the URL.  And I'll find a link to the raw study when I'm able.   

So, go ahead -- tell me there's another study out there, and it really and truly proves that vouchers are the greatest thing since sliced bread.  But don't expect me to click on any more of your links.  Playing whack-a-mole with a series of flawed studies and broken links isn't my idea of fun.  

This actually dovetails into something I noted about Jay Mathews column from yesterday.  But on the way there, let's talk about Matthew Ladner's steak dinner challenge.  As he noted at the time:

Random assignment control-group studies, the gold standard of social science research, allow us to stop armchair theorizing. When done properly, these studies allow researchers to isolate the impact of a particular program by observing two groups that are effectively identical.

In a very real sense, these sorts of studies are rather similar to FDA clinical drug trials.  The structure of the trials can vary a great deal, but I think most people grasp their general framework: two groups of people with the same condition are assembled, one group (the experimental group) is administered the new drug, the other group (the control group) is basically left alone, and group members are monitored for changes in their condition to examine the drug's effect.  Of course, there are certain aspects of such experiments that just can't be replicated in the environment of education, such as double-blind studies and administration of placebos, but the fundamental structure of an experimental group and control group is intact. 

Ladner goes on to point out that school choice programs have been subjected to several such random-assignment control group studies, and that the programs have consistently been shown to improve student performance.  None has been shown to inflict academic damage.  However, the body of information is routinely rejected by school choice opponents as flawed or biased.  It's not likely that this will change anytime soon--there's always an academic somewhere who will disagree over the strength of a particular study, and allegations of slant in research will probably always be with us. 

But in that light, what I find profoundly disturbing is that, for ed schools, it's research and only research--and, for that matter, only the right kind of research--that defines the proper approach to education.  We now arrive at Jay Mathews's column:

I asked why our education schools did not teach the many practical and effective methods of teaching in the inner city developed by our best instructors. I cited examples from the playbooks of four nationally renowned educators, Rafe Esquith, Mike Feinberg, Dave Levin and Jason Kamras. Each of them had much more experience with low-income kids than the average ed-school professor, and their methods -- none of them learned in ed school -- had helped produce exceptional gains in student achievement.

I focused on their most unusual and provocative approaches, the ones least likely to reach the ivory towers of our best teacher-training institutions. Kamras, Feinberg and Levin, for instance, had success making unannounced visits to inner city parents who could not be reached on the phone or by e-mail. Esquith invented a system that paid low-income Los Angeles fifth graders virtual dollars based on their work and boosted their understanding of math, economics and geography. Kamras gave his D.C. students quick feedback on homework by only grading a few of their answers. Feinberg and Levin required students to call their teachers at home if they had homework questions.

Many of the education school people said that as interesting as such methods were, they could not teach them until they had been verified by research. I protested that researchers usually chewed such lessons into indigestible mush, with conclusions too vague or too controversial to help beginning teachers.

It's at this point that you really can't blame many edureformers for crying uncle.  Experienced teachers, working in the trenches, discover that certain techniques and methods generate results--but ed school professors refuse to accept said techniques as sound until they see some sort of study as verification.  It's almost as if Moses must descend from the mountain before certain academics will believe. 

Research is critically important.  Good studies can certainly be vital.  And as noted before, academics are constantly haggling over minute details (hey, minute details are their stock in trade, as Jesse Rothstein will attest).  But it's increasingly apparent that, when it comes to education policy, practically no research--even common sense observation--is good enough.  Parental feedback on what works for their kid?  Insufficient.  The work of Harvard professors like Paul Peterson and Caroline Hoxby?  Slanted.  And if even nationally regarded teachers can't be taken at their word, somehow I doubt we in the school choice community can expect a fair shake.  In fact, here's a question for John: do you consider any study favorable to school choice to be even minimally reliable or trustworthy?

One last thing.  There are no perfect studies out there--pretty much all education research has deficiencies, however small.  But a significant body of very solid evidence demonstrates that school choice programs benefit students and their families.  And school vouchers aren't "the greatest thing since sliced white bread".  But I continue to assert that giving parents options for the education of their children is the best way to benefit both students and their schools, particularly those children presently in failing public schools with no recourse.  If that's playing "whack-a-mole", then guilty as charged.