• The Heritage Network
    • Resize:
    • A
    • A
    • A
  • Donate
  • Media Matters Tries but Fails to Refute the School Choice Evidence

    schoolchoicesign

    Yesterday, Media Matters tried to refute a blog post in which I point out, among other things, that the impact of voucher use in the D.C. Opportunity Scholarship Program resulted in a 91 percent graduation rate compared to 70 percent in the control group. The findings are from the U.S. Department of Education’s final evaluation of the voucher program, authored by Dr. Patrick Wolf.

    Walid Zafar writes via Media Matters:

    Where does Burke get the 91 percent figure from? Well, not this [the Department of Education’s] report. It’s hard not concluding that she made that statistic up. The report puts the graduation rate for students receiving vouchers at 82 percent.

    Not so fast. The report does in fact find that the use of voucher resulted in a 91 percent graduation rate. On page 20 of the report’s executive summary, Wolf writes:

    The offer of an OSP scholarship raised students’ probability of completing high school by 12 percentage points overall. The graduation rate based on parent-provided information was 82 percent for the treatment group compared to 70 percent for the control group. There was a 21 percent difference (impact) for using a scholarship to attend a participating private school. [Emphasis added]

    The 21 percentage point difference for impact means the typical student who received a voucher and actually used it to attend a private school had a graduation rate of 91 percent, compared to 70 percent for non-voucher students. Here’s exactly how the graduation rates break down:

    • D.C. Public Schools graduation rate: 49 percent.
    • Control group (those students who applied for a voucher but did not receive one) graduation rate: 70 percent.
    • Voucher recipient group (students who applied for a voucher, won the lottery to receive one, but did not necessarily use it) graduation rate: 82 percent.
    • Impact of voucher use: (students who applied for, received, and actually used the voucher to attend a private school) graduation rate: 91 percent.

    Zafar also argues that the results of the study are minimized due to the increased motivation of parents who applied for a scholarship:

    You can’t compare the graduation rate at DC Public Schools (which take in all who apply, regardless of learning disabilities and level of parental involvement) to a lottery based voucher system to which only the most highly motivated students (and parents) choose to apply.

    First, the D.C. Opportunity Scholarship Program does have to take all students who apply. When applications exceed scholarships, officials use a lottery to determine which students receive vouchers. In fact, because evaluators anticipated objections like Zafar’s, they controlled for the students who applied for a voucher but were ultimately not offered one. These presumably highly motivated students were evenly distributed across the treatment and control group, which is probably why the control group graduation rate of 70 percent was higher than the overall DCPS rate of 49 percent.  The voucher students significantly outperformed the control group on the crucial measure of high school graduation even though the lottery ensured that both groups were equally stocked with motivated students and parents.

    While it’s true that parents have to have a certain level of interest in the educational opportunities of their children in order to apply for a voucher, thousands of low-income families in the District jumped at the opportunity to do so when given the chance. In fact, there were four applicants for every available scholarship.

    Finally, Zafar argues that the DCOSP had no impact on academic achievement:

    In the area of student achievement, the report concludes, “Overall reading and math test scores were not significantly affected by the Program, based on our main analysis approach.” Most crucially, the report notes that “No significant impacts on achievement were detected for students” who “were lower performing academically when they applied.” In other words, the students who did well on the voucher program were those who were already doing well in public school.

    While the final evaluation did not find a statistically significant impact on academic achievement (which was not the main point of our argument), it did find that the scholarships had a positive impact on academic outcomes for some subgroups of students. Moreover, Dr. Wolf, the lead researcher on the OSP study, explains in a statement from the University of Arkansas that the significant positive impact on graduation rates is more important than the impact on academic achievement:

    These results are important because high school graduation is strongly associated with a large number of important life outcomes such as lifetime earnings, longevity, avoiding prison and out-of-wedlock births, and marital stability. Academic achievement, in contrast, is only weakly associated with most of those outcomes.

    In the area of education, how far you go is more important than how much you know, and D.C. students went farther with the assistance of a school voucher.

    Facts matter, and we hope we’ve stated them clearly enough so that even Media Matters can’t deny them.

    Posted in Education [slideshow_deploy]

    33 Responses to Media Matters Tries but Fails to Refute the School Choice Evidence

    1. Simon, Seattle says:

      Minor point: was it a 21 percentage point increase, or a 21 percent increase? A 21 percent increase would just raise it to 85%, assuming I did my math right.

    2. Brandi Martin says:

      This is a good point, however remember the students who were unmotivated to apply are still back in the old schools, where they require more resources, time, redirection, remediation and follow up calls, all of which siphons away some of the teachers time. I am presuming teachers at the voucher schools had more time for instruction, and one on one with the kids who are actually doing homework and classwork. I'm sure it makes a difference. However these kids have to go somewhere. Give them a voucher and bus them over and that 91% rate will sink like a rock.

    3. Mike, Maryland says:

      Facts do matter, and when researchers fudge them to promote a policy agenda, they should get called out, as should think tanks that echo the rhetoric.

      The study's claims are not based on actual graduation rates, but rather on parent reports of student educational attainment. How hard would it have been for the researchers to use actual rather than anecdotal data, and why did they choose the latter over the former?

      Furthermore, we're only talking about 19 actual students who used the vouchers for the full five years of the program and were old enough when they entered to have graduated by or before the program conclusion. How statistically generalizable are the results?

      Finally, the study asserts that there was no statistically significant evidence of impact among students from schools not identified as needing improvement or for students who entered the program with relatively low levels of academic performance (the purported targets of the program), or for boys. To reach the conclusion that the program enhanced the chances of graduating from high school, researchers had to average that statistical insignificance with the greater impact the program allegedly had on graduation rates among high achievers (who would have graduated anyway) and girls.

      Rick Hess at AEI recently cautioned school choice advocates about over-promising. This defense of the DC voucher program appears not to have heard that advice. It should make one wonder why such an effort is being made to defend a program that has not proven effective for all students receiving the treatment. Is it perhaps that facts don't really matter when you're pushing an agenda?

    4. Pingback: Must Know Headlines 10.15.2010 — ExposeTheMedia.com

    5. Drew Page, IL says:

      The only media that matters is honest media and that excludes about 90% of the media, especially Media Matters.

    6. John Clancy says:

      The comments above seem to go round and round with respect to looking at a way to impove the present situation. They refer to points that ultimately diminish or negate the value of choice. We're left, then, with the same monolithic system that fails us.

      When you step back, however, from a system of education and look at other areas of life that are relatively complex but successful–medical research, health delivery systems, engineering the automobile, areas that include competition within–wisdom might direct you to diversify our present educational system that is monolithic and disproportionaely expensive.

    7. Lindsey Burke Lindsey says:

      Yes, it is a percentage point increase.

    8. Bob Rose, Jasper, Ge says:

      THE WSJ ANSWER TO SCHOOL REFORM:

      This article appeared on-line on Oct 5, 2010. The URL for the article is

      online.wsj.com/article/SB10001424052748704631

      How Handwriting Trains the Brain

      Forming Letters Is Key to Learning, Memory, Ideas

      Article

      Video

      Comments (26)

      more in Business »

      Email

      Print

      Save This ? More

      Twitter

      Digg

      + More

      close

      Yahoo! Buzz

      MySpace

      del.icio.us

      Reddit

      Facebook

      LinkedIn

      Fark

      Viadeo

      Orkut

      Text

      By GWENDOLYN BOUNDS

      Ask preschooler Zane Pike to write his name or the alphabet, then watch this 4-year-old's stubborn side kick in. He spurns practice at school and tosses aside workbooks at home. But Angie Pike, Zane's mom, persists, believing that handwriting is a building block to learning.

      Wendy Bounds discusses the fading art of handwriting, pointing out that new research shows it can benefit children's motor skills and their ability to compose ideas and achieve goals throughout life.

      More Health Videos

      NanoKnife Zaps Tumors With Electricity (10/4/10)

      Laser Beams That Zap Your Fat Away! (09/15/10)

      Why Gluten-Free Isn't for Everyone (08/23/10)

      Audio

      Gwendolyn Bounds reports on what your handwriting says about your brain and everything else.

      She's right. Using advanced tools such as magnetic resonance imaging, researchers are finding that writing by hand is more than just a way to communicate. The practice helps with learning letters and shapes, can improve idea composition and expression, and may aid fine motor-skill development.

      It's not just children who benefit. Adults studying new symbols, such as Chinese characters, might enhance recognition by writing the characters by hand, researchers say. Some physicians say handwriting could be a good cognitive exercise for baby boomers working to keep their minds sharp as they age.

      Studies suggest there's real value in learning and maintaining this ancient skill, even as we increasingly communicate electronically via keyboards big and small. Indeed, technology often gets blamed for handwriting's demise. But in an interesting twist, new software for touch-screen devices, such as the iPad, is starting to reinvigorate the practice.

      View Full Image

      Angie Pike

      Four-year-old Zane Pike used to toss aside his handwriting books. Now, the Cabot, Ark., preschooler is learning to write his letters using a smartphone application.

      Most schools still include conventional handwriting instruction in their primary-grade curriculum, but today that amounts to just over an hour a week, according to Zaner-Bloser Inc., one of the nation's largest handwriting-curriculum publishers. Even at institutions that make it a strong priority, such as the private Brearley School in New York City, "some parents say, 'I can't believe you are wasting a minute on this,'" says Linda Boldt, the school's head of learning skills.

      Recent research illustrates how writing by hand engages the brain in learning. During one study at Indiana University published this year, researchers invited children to man a "spaceship," actually an MRI machine using a specialized scan called "functional" MRI that spots neural activity in the brain. The kids were shown letters before and after receiving different letter-learning instruction. In children who had practiced printing by hand, the neural activity was far more enhanced and "adult-like" than in those who had simply looked at letters.

      "It seems there is something really important about manually manipulating and drawing out two-dimensional things we see all the time," says Karin Harman James, assistant professor of psychology and neuroscience at Indiana University who led the study.

      More

      The Juggle: In Digital Age, Does Handwriting Still Matter?

      Adults may benefit similarly when learning a new graphically different language, such as Mandarin, or symbol systems for mathematics, music and chemistry, Dr. James says. For instance, in a 2008 study in the Journal of Cognitive Neuroscience, adults were asked to distinguish between new characters and a mirror image of them after producing the characters using pen-and-paper writing and a computer keyboard. The result: For those writing by hand, there was stronger and longer-lasting recognition of the characters' proper orientation, suggesting that the specific movements memorized when learning how to write aided the visual identification of graphic shapes.

      Other research highlights the hand's unique relationship with the brain when it comes to composing thoughts and ideas. Virginia Berninger, a professor of educational psychology at the University of Washington, says handwriting differs from typing because it requires executing sequential strokes to form a letter, whereas keyboarding involves selecting a whole letter by touching a key.

      She says pictures of the brain have illustrated that sequential finger movements activated massive regions involved in thinking, language and working memory—the system for temporarily storing and managing information.

      And one recent study of hers demonstrated that in grades two, four and six, children wrote more words, faster, and expressed more ideas when writing essays by hand versus with a keyboard.

      View Full Image

      AJ Mast for the Wall Street Journal

      For research at Indiana University, children undergo specialized MRI brain scans that spot neurological activity.

      Even in the digital age, people remain enthralled by handwriting for myriad reasons—the intimacy implied by a loved one's script, or what the slant and shape of letters might reveal about personality. During actress Lindsay Lohan's probation violation court appearance this summer, a swarm of handwriting experts proffered analysis of her blocky courtroom scribbling. "Projecting a false image" and "crossing boundaries," concluded two on celebrity news and entertainment site hollywoodlife.com. Beyond identifying personality traits through handwriting, called graphology, some doctors treating neurological disorders say handwriting can be an early diagnostic tool.

      "Some patients bring in journals from the years, and you can see dramatic change from when they were 55 and doing fine and now at 70," says P. Murali Doraiswamy, a neuroscientist at Duke University. "As more people lose writing skills and migrate to the computer, retraining people in handwriting skills could be a useful cognitive exercise."

      In high schools, where laptops are increasingly used, handwriting still matters. In the essay section of SAT college-entrance exams, scorers unable to read a student's writing can assign that portion an "illegible" score of 0.

      Even legible handwriting that's messy can have its own ramifications, says Steve Graham, professor of education at Vanderbilt University. He cites several studies indicating that good handwriting can take a generic classroom test score from the 50th percentile to the 84th percentile, while bad penmanship could tank it to the 16th. "There is a reader effect that is insidious," Dr. Graham says. "People judge the quality of your ideas based on your handwriting."

      Handwriting-curriculum creators say they're seeing renewed interest among parents looking to hone older children's skills—or even their own penmanship. Nan Barchowsky, who developed the Barchowsky Fluent Handwriting method to ease transition from print-script to joined cursive letters, says she's sold more than 1,500 copies of "Fix It … Write" in the past year.

      Some high-tech allies also are giving the practice an unexpected boost through hand-held gadgets like smartphones and tablets. Dan Feather, a graphic designer and computer consultant in Nashville, Tenn., says he's "never adapted well to the keypads on little devices." Instead, he uses a $3.99 application called "WritePad" on his iPhone. It accepts handwriting input with a finger or stylus, then converts it to text for email, documents or Twitter updates.

      And apps are helping Zane Pike—the 4-year-old who refused to practice his letters. The Cabot, Ark., boy won't put down his mom's iPhone, where she's downloaded a $1.99 app called "abc PocketPhonics." The program instructs Zane to draw letters with his finger or a stylus; correct movements earn him cheering pencils.

      Indiana University

      In children who had practiced writing by hand, the scans showed heightened brain activity in a key area, circled on the image at right, indicating learning took place.

      "He thinks it's a game," says Angie Pike.

      Similarly, kindergartners at Harford Day School in Bel Air, Md., are taught to write on paper but recently also began tracing letter shapes on the screen of an iPad using a handwriting app.

      "Children will be using technology unlike I did, and it's important for teachers to be familiar with it," says Kay Crocker, the school's lead kindergarten teacher. Regardless of the input method, she says, "You still need to be able to write, and someone needs to be able to read it."

      Write to Gwendolyn Bounds at wendy.bounds@wsj.com

    9. Edd Doerr, Maryland says:

      This article does not mention that in their only opportunity to vote on this matter, District voters in 1981 defeated a tax-code voucher scheme by 89% to 11%, that school voucher plans are of dubious constitutionality and do violate the constitutions of at least 38 states, that tens of millions of voters in over 25 statewide referenda fron coast to coast have rejected vouchers or their vafiants by two to one, and that Congress thus year wisely voted to end the DC voucher plan.

    10. Matt, Alabama says:

      The link to the earlier post ("Michelle Rhee Leaves A Legacy of Reform) is broken. Correct link is here:
      http://www.foundry.org/2010/10/14/media-matters

    11. Pingback: ADF Alliance Alert » Heritage Foundation: Media Matters Tries but Fails to Refute the School Choice Evidence

    12. Spiritof76, NH says:

      Whether it produces in DC a night-day contrast as a result of vouchers is immaterial. The student and his/her parents must determine the outcome by having the ability to exercise their option. After all, that is what is meant by opportunity. It is not opportunity for the most probable outcome of failure.

      Sweden (that great nation of welfare)has instituted a school voucher program since the 1990s with great success and satisfaction for the customers – students.

      The only reason there is so much demagoguery is because of the special interest group , called the teachers union, that is feeding on the tax dollars. Let us be honest about it. The union doesn't care about the students achieving success, They are only the vehicle for the confiscation of tax dollars.

    13. G. Charles, Detroit says:

      I see Lindsey affirmed it was a 21% POINT increase, not just a percent. This was an important point so I double-checked the study and three times it explicitly states this:

      "The offer of an OSP scholarship raised students’ probability of graduating by 12 percentage points and the use of a scholarship by 21 percentage points"

      "Overall, a positive impact on high school graduation of 12 percentage points for students who received the offer of a scholarship. The high school graduation rate was 82 percent for the treatment group compared to 70 percent for the control group. Using a scholarship increased the graduation rate by 21 percentage points." [That is, 70 + 12 percentage points = 82; 70 + 21 percentage points = 91.]

      "The impact on graduation of actually using a scholarship to attend private school was 21 percentage points. "

      Also, Table 3:5 of the study footnotes that the 21 percent increase is "statistically significant at the 99 percent confidence level."

    14. iccc, chicago says:

      Scholarships, by all means. As long as the poor kids are not attending the pricey school that the president's kids are attending.

      Wonder how many scholarships the Lady has spent on her Antoinnette tour? $375,000?

    15. Pingback: Instapundit » Blog Archive » HERITAGE 1, MEDIA MATTERS 0: A fish, a barrel, a smoking gun….

    16. dennymack, Oregon says:

      So the voucher program showed modest gains in its first few years. I wonder why it was so very important to kill the program? While it sounds like a lot of money, it is a drop in the bucket compared to the D.C. system as a whole. The "siphoning off resources" argument assumes that the public system is lacking resources. Not a factor: they get far more than most districts that do far better.

      What gains has the D.C. public school system shown in the last five years? Ten years? Twenty?

      D.C. had to get rid of vouchers, and Rhee, because they had real potential to show that improvement is possible, if one changes the status quo. Nowhere in the world is the status quo more thoroughly defended than in our biggest, worst school districts.

      So tell the parents to be patient. We are aware of the problem and working on it.

    17. Dave M., S. Korea says:

      Yes Edd, 29 years ago the D.C. folks voted it down so, since absolutely nothing has changed since then, why give them another chance. Constitutionally dubious? That assertion is rather dubious. I cannot find anywhere in the Constitution where states are not allowed to have voucher programs. Wait, I'll look again. Nope, still cannot find it.

      Edd, I guess you are one of those compassionate leftist. Whether something works or not does not matter to you. You only care that government gets bigger and bigger and citizens get smaller and smaller. Stay away from Virginia Edd, we have enough "good intention" leftists in Arlington. We do not need anymore.

    18. Doug says:

      You have to admire the commitment to mediocrity that the left embodies. No data, no fact, and certainly no negative outcomes for our children are enough to deter them from their appointed tasks.

    19. datechguy says:

      Since media matters is a political organization funded for the express purpose to enforce a political agenda there is no level of fact too high that Media Matters can not deny them.

    20. Cincinnatus says:

      Correct me if I’m wrong: This is the program that Obama shut down as soon as he became President?

    21. Joe, Maryland says:

      Once again the leftists are allowed to shape and define the argument by ignoring the real point. The point of school choice folks, is that parents (consumers) will be able to choose the school their children attend thus putting market pressure on underperforming schools to improve. I know this is hard for idiot leftists to understand, but its called competition. A tiny pilot program with just a few participants puts zero pressure on underperforming schools and therefore demonstrates NOTHING. American public schools are a tragedy. If we continue to allow institutionalized centralized government and union controlled power in our nation we will reap the bitter fruit soon enough.

    22. Jamie, Pennsylvania says:

      What matters to you, Edd Doerr? If graduation rates (49%? That's a SCANDAL) and therefore life outcomes could be boosted by some means that is either already legally fine or could be made legally acceptable by voters (or, I suppose, by an enlightened judiciary with no dog in the teachers' union fight), would you support that means? This article sets out to answer Media Matters' claim that school vouchers had no real effect on students in the DC system. Apparently they had some pretty important effect on those who used them – a small subgroup, but a subgroup that now is almost entirely composed of HIGH SCHOOL GRADUATES.

      What's your argument? That the vouchers had no effect, as Media Matters would like us to believe, or that they're morally wrong because there are not enough of them for all students, as the commenter Brandi Martin seems to be saying, or that they shouldn't be used because they're illegal and should remain illegal forever because… um… because constitutions shouldn't be amended and voters cannot change their minds? Data help us, if our aim is to improve the status quo. Data are our enemy, if our aim is to maintain the status quo because of whatever benefits we gain specifically from it.

      Are you a public school teacher, by any chance?

      I haven't made up my mind about school vouchers; my children attend a public school, but it's in a district where graduation rates are quite high, and my husband and I are motivated parents with children who do not need special services. How would I feel if any of these variables were different? I'm not sure, but data are my friend as I try, as a voter, to decide.

    23. Pingback: Media Matters Tries but Fails to Refute the School Choice Evidence

    24. Harry Berber, Pleasa says:

      Who pays Zafar to write that crap?

      Evil "foreign money", no doubt.

    25. Ted Mauro, SC says:

      1.There is no conclusive evidence that the OSP affected student achievement.

      Although some other subgroups of students appeared to have higher levels of reading achievement if they were offered or used a scholarship, those findings could be due to chance.

      2.The Program significantly improved students’ chances of graduating from high school. Although students may not have raised their test scores in reading and math as a result of the OSP, they graduated at higher rates. The offer of an OSP scholarship raised students’ probability of completing high school by 12 percentage points overall (figure ES-3). Causation can not be concluded from this report because once again we have no idea what could explain this, we do not have any other reports or studies to link to it and the numbers were too small.

      3.The OSP raised parents’, but not students’, ratings of school safety and satisfaction (figures ES-4 and ES-5). Parents were more satisfied and felt school was safer if their child was offered or used an OSP scholarship. The Program had no effect on students’ reports on school conditions.

      4.Test score correlation was not factored into the power forecasts for the attainment analysis. The use of test scores and completion are just SOME measures per NCLB. We call upon the authors to use a full comparision

      5 Finally – what is diffrent between those who don't take the voucher and those who did- not academics so what is it and could that explain the diffrence?

    26. Kangpin says:

      Wow, MMFA has issued an update to the post, "The Department of Education study did in fact note a 21 percent difference between those who used the vouchers and the control group…We apologize for making that mistake." Shocking, pure shocking. Too bad Soros will probably fire Zafar for acknowledging/pretending facts matter.

    27. Troy, Mesa Arizona says:

      Vouchers may or may not work. Clearly, there is room for dispute. Here in Arizona, my kids go to an excellent Charter School which consistently scores in the 90% on the AIMS test. Public schools in our area score below 70% consistently. AND we don't have to worry about them teaching our Kindergartners how to use a condom.

    28. OldSalt - Washington says:

      Pierce v. Society of Sisters of the Holy Names of Jesus and Mary, (1925), where the unanimous court found that, "the fundamental liberty upon which all governments in this Union repose excludes any general power of the State to standardize its children by forcing them to accept instruction from public teachers only."

    29. Aebe, Orrigone says:

      Having shown that vouchers being used at charter schools in D.C. were working,of course the President had to put an end to that.

      back to a system where less than 50% of students graduate,youbetcha.

      You done good,Osama,now go out and booger up someone else' future.

    30. Pingback: Saving Money Through School Choice | The Foundry: Conservative Policy News.

    31. Pingback: The Corner | Arkansas News | Arkasas Breaking News Headlines | Arkansas News Directory

    32. Pingback: Why school vouchers are worth a shot | School Choice Indiana

    33. Pingback: Give school vouchers a chance « NNY912

    Comments are subject to approval and moderation. We remind everyone that The Heritage Foundation promotes a civil society where ideas and debate flourish. Please be respectful of each other and the subjects of any criticism. While we may not always agree on policy, we should all agree that being appropriately informed is everyone's intention visiting this site. Profanity, lewdness, personal attacks, and other forms of incivility will not be tolerated. Please keep your thoughts brief and avoid ALL CAPS. While we respect your first amendment rights, we are obligated to our readers to maintain these standards. Thanks for joining the conversation.

    Big Government Is NOT the Answer

    Your tax dollars are being spent on programs that we really don't need.

    I Agree I Disagree ×

    Get Heritage In Your Inbox — FREE!

    Heritage Foundation e-mails keep you updated on the ongoing policy battles in Washington and around the country.

    ×