Cherry-Picking Michigan Charter Data Leads to Wrong Conclusions

The National Cherry Festival in Traverse City is more than four months away, but when it comes to the record of charter schools, Education Trust-Midwest has started picking early.

Ed Trust tried to stake a claim in the debate about Michigan's lackluster educational performance by aiming squarely at a small sector of public education: charter schools. It's appropriate to have a discussion about what more charter schools could do to help drive improvements. And reasonable observers can express frustration that charters haven't raised the bar high enough.

But to argue that Michigan charters are doing worse than other public schools and are “a national embarrassment” is a gross misrepresentation of what the research evidence actually says.

Ed Trust's latest report states: “According to the Center for Research on Education Outcomes (CREDO) at Stanford University, roughly 80 percent of charters have academic achievement below the state average in reading and math.” The editors of the Detroit Free Press uncritically repeated the statement. Amber Arellano, one of the co-authors of the Ed Trust report, landed on the same point in a Huffington Post op-ed.

There's only one problem: This is a cherry-picked, simplistic statistic that runs exactly counter to the CREDO study’s actual findings. The isolated statement is drawn out of the context of a larger report that strenuously analyzes student-level data over a five-year period. The results of this sophisticated analysis make a very different point: Most Michigan charters are performing above the state average. The report found that 82 percent of Michigan charters created higher than average growth in reading, and 72 percent had higher growth in math.

These robust results take into consideration the fact that charter schools, on average, serve a much larger share of low-income and low-performing students compared to the rest of Michigan schools. In fact, nearly all of the CREDO study’s findings take this into account, except the one statistic that Ed Trust and Freep editors regularly repeat. Apparently, they’d prefer to ignore this reality and analyze apples in order to understand oranges.

What’s more, Ed Trust’s Arellano touts Massachusetts as a charter school success story — an alternative to Michigan’s supposed failure. But CREDO researchers did the same type of analysis of Massachusetts’ charter schools as they did of Michigan’s. And the findings throw a wrench into her argument. Michigan charters on average registered a slightly better track record in helping students improve reading compared to those in Massachusetts, while Massachusetts charters did slightly better in math.

There is one difference, though: CREDO found 44 percent of Massachusetts charters outperforming conventional schools in reading and 56 percent outperforming in math. In Michigan, 35 percent of charters bested traditional schools in reading and 42 percent posted similar results in math. But Michigan had a smaller percentage of charter underperforming compared to Massachusetts. Only 2 percent of Michigan charters underperformed in reading and only 6 percent in math. In Massachusetts, these figures were 13 and 17 percent, respectively.

Based on this type of school-to-school comparison, the CREDO results show that almost all charter schools in Michigan are doing about the same or better than their conventional school counterparts. Hardly an embarrassment, or, as Arellano puts it, “a national poster child of how not to do charter schools.”

In fact, CREDO made just the opposite point about Detroit charter schools in a different study of charter performance in urban communities. CREDO researchers identified Detroit as one of four cities in the country that had both a small share of low-performing charters and where a majority of charters outperformed their conventional counterparts. The report states: “These four communities of charter schools provide essential examples of school-level and system-level commitments to quality that can serve as models to other communities.”

Based on the best evidence we have, the type that measures student learning growth and controls for poverty and other socioeconomic factors, charter schools are a net benefit to public education in this state. Those who would suggest otherwise need to provide better evidence than a cherry-picked and crude statistic.