Beth Zirbes: Analysis of the Harvard Charter School Study
Beth Zirbes is a math teacher in Alaska, and when she saw the Harvard study claiming great things for charters, she wondered if the results would hold up. What did she find?
At first glance, it appears that charter schools are more successful than neighborhood schools. At charter schools 52.5% (1,866 out of 3,554) of students were proficient on the ELA assessment versus 40.1% (18,655 out of 46,574) of students at neighborhood schools. However, these differences could be explained by the differences in demographics of the student bodies at these schools. To rule this out as a potential issue, statisticians control for these variables in their mathematical models. We can then ask, do the charter schools outperform neighborhood schools that have similar characteristics? Or do charter schools do any better than we would expect, given their student populations? In short, the answer is no, they do not. When I fit a model which controlled for socioeconomic status alone, the type of school (charter versus neighborhood) was not significant. (For anyone who knows statistics, the p-value associated with type of school was 0.57. It wasn’t even close.) In summary, there is no evidence that charter schools outperform neighborhood schools in terms of ELA proficiency once we consider their socioeconomic make-up.
During my data exploration, I discovered that charter schools, on average, have very different student bodies than neighborhood schools. Charter schools have far fewer economically disadvantaged students, far fewer ELL students, and are comparable to neighborhood schools in terms of SPED populations. Here is a summary of how these populations differ for all Alaskan students in the relevant grades in all of Alaska’s brick and mortar schools for the 2018-2019 school year:
- Neighborhood schools were 52.2% economically disadvantaged (30,780 out of 58,929 students) compared to 31.3% in charter schools (1,219 out of 3895 students).
- Neighborhood schools were 15.5% ELL (9,150 out of 58,929 students) compared to 9.3% in charter schools (363 out of 3895 students).
- Neighborhood schools were 16.3% SPED (9,162 out of 58,929 students) compared to 13.7% in charter schools (532 out of 3895 students).
However, these summaries are highly influenced by a few outliers and obscure some large discrepancies, especially in terms of the economically disadvantaged and ELL students.
- Of the charter schools, 46.4% (13 out of 28) have economically disadvantaged rates below 20%, compared to just 3.5% of neighborhood schools (15 out of 426).
- Only 10.7% of charter schools (3 out of 28) have ELL percentages above 10%, compared to 36.9% of neighborhood schools (157 out of 426).
Read her full piece at the Association of Alaska School Boards web site.