Thomas Ultican: Propaganda from The 74 and University of Arkansas
Blogger Thomas Ultican responds to a recently released piece of “research” about charter school “productivity.” Reposted with permission.
The 74 published a blatant propaganda piece on Monday (11/20/2023), based on “Still a Good Investment: Charter School productivity in Nine Cities,” a paper from the University of Arkansas’s “School Choice Demonstration Project (SCDP).” In this production, SCDP used its own previously debunked work to support ridiculous conclusions.
The Department of Education at University of Arkansas does not attempt to hide their anti-public school bias, as noted in the cited paper, “The College of Education and Health Professions established the Department of Education Reform in 2005.” Subsequently, “The School Choice Demonstration Project” was established and staffed by “leading school choice researchers and scholars” within the Department of Education Reform. (Page 3)
The paper’s executive summary states:
“In this study, we reexamine the productivity of publicly funded schools, using funding data from our charter school revenue report ‘Charter School Funding: Little Progress Towards Equity in the City.’ We also use achievement data from the Center for Research on Educational Outcomes’ (CREDO’s) city and national studies, the NAEP Data Explorer, and wage data from the Bureau of Labor Statistics. We have access to complete data for nine cities: Camden, New Jersey; Denver, Colorado; Houston, Texas; Indianapolis, Indiana; Memphis, Tennessee; New Orleans, Louisiana; New York City, New York; San Antonio, Texas; and Washington, DC.” (Page 4)
Phony Financial Data
The 74 used the following graphic to open their propaganda piece:
That $8,000 less per student claim is based on a previous SCDP study, “Charter School Funding: Little Progress Towards Equity in the City.” This September, researcher Mark Webber from Rutgers University posted at the National Education Policy Center:
“The following problems have been repeatedly pointed out by disparate third-party reviewers. Yet there appears to be little or no willingness to move away from the flawed approaches, which continue to plague report after report.”
- Inadequate documentation of data
- Misunderstanding of financial transfers
- Invalid conflation of individual schools and school districts as units of analysis
- Invalid comparisons of student populations
- Invalid comparisons of the functions of charter and district public schools
- Unaccounted-for charter revenues
- Neglect of the literature on charter school finances
In 2019, David S. Knight, University of Washington and Laurence A. Toenjes, University of Houston wrote “Do Charter Schools Receive Their Fair Share of Funding? School Finance Equity for Charter and Traditional Public Schools.” By focusing on Texas, they demonstrated how difficult this question is and that the answer showed no significant difference.
In 2021, school finance expert, Bruce Baker reported:
“A report from the University of Arkansas Department of Education Reform asserts that charter schools, despite serving only marginally fewer children with disabilities than traditional public schools, are significantly shortchanged of funding for those children, in addition to being significantly shortchanged on funding in general. This assertion is erroneous because the report ignores substantial differences in the classifications, needs, and costs of children with disabilities in district-operated versus charter schools. To reach its incorrect conclusions, the report exclusively self-cites deeply flawed, self-published evidence of a general charter school funding gap, ignoring more rigorous studies yielding contradictory findings. The report adds no value to legitimate debate over the comparability or adequacy of general or special education funding of charter schools.”
Fraudulent Testing Data
The SCDP report says, “Based on CREDO’s findings, we estimate that charter school students across nine cities perform 2.4 points (0.06 standard deviations, or SD) higher on the eighth grade reading NAEP exam and 1.3 points higher (0.03 SD) on the math exam, compared to matched TPS students.” (Page 5)
There are reasons to believe the CREDO findings are bunkum. They have exclusive access to the data they report on and their methodology is highly suspect. None of these studies are submitted for peer review.
CREDO is the Center for Research on Educational Outcomes, a part of the Hoover Institute at Stanford University in California. The Institute is a conservative center funded by groups like the Walton Family Foundation, a key part of the radical conservative movement to end our traditional public school system.
Macke Raymond is the head of CREDO. Her 2015 Hoover Institute Fellow’s profile says, “In partnership with the Walton Family Foundation and Pearson Learning Systems, Raymond is leading a national study of the effectiveness of public charter schools.”
The Technical Appendix for the “Cities Studies Project” reports, using growth models without identifying which model and says:
“In our study, scores for all these separate tests are transformed to a common scale. All test scores have been converted to standardized scores to fit a ‘bell curve’, in order to allow for year-to-year computations of growth.”
The “Education Growth Model Handbook” lists seven types of growth models in general use and their requirements. Most growth models require vertical scales but that does not seem possible with CREDO’s use of multiple tests, many of which are not vertically scaled. Their mathematical conversions add a locus of error.
CREDO’s method does not compare charter school performance to actual public schools; rather, it creates mathematical simulations, called “virtual twins.” Business writer, Andrea Gabor, hired statistician, Kaiser Fung, to help explain the bias, inherent in CREDO’s approach. She reported that the “study excludes public schools that do NOT send students to charters, thus introducing a bias against the best urban public schools, especially small public schools that may send few, if any, students to charters.”
Professor Andrew Maul of UC Santa-Barbara noted when writing about a 2015 paper, “The study’s ‘virtual twin’ technique is insufficiently documented, and it remains unclear and puzzling why the researchers use this approach rather than the more accepted approach of propensity score matching.”
Earlier this year, Network for Public Education Director, Carol Burris, published “In Fact or Fallacy? An In-Depth Critique of the CREDO 2023 National Report,” stating “The virtual twin matching methodology gives rise to a second issue: the exclusion of about one in five charter schools due to a lack of a match in feeder public schools for charter school students.” (Page 6)
CREDO with its fancy math found that charter schools in the nine cities they studied outperformed public schools. However, there is no way to check the results since only they have access to the data. The graphs below were created by staff at the National Assessment of Education Progress, comparing eighth grade math and reading results for charter schools and public schools.
These are graphs of raw data, separated by type of school. Charter schools never outperformed public schools, making one wonder about CREDO’s results.
Conclusion
Since financial inputs and test-scores were determinative in this report, the rest of the report is just icing on a baloney cake. Even if based on pristine data, estimates of “lifetime earnings” are meaningless.
Patrick Wolf and his team should be embarrassed.
When the University of Arkansas puts out a study like this, it is amplified through rightwing media. The Center for Education Reform immediately posted an article, praising the recent work. The Indiana Capital Chronicle wrote how Indiana needs to shift more money away from public schools, based on this “research”. Epoch Times, The 74 and NJ Education Report all praise the Arkansas paper. Search engines also show a long list of links from the University of Arkansas and paper depositories where they upload their work.
If there is any push back, it would be an article from the National Education Policy Center or maybe something posted by Diane Ravitch.
It is interesting the choice industry has succumbed to lying, to make their case. The public school system is sound and taking it down while profiting is not happening.
This latest SCDP report is straight-up fraud.