Are there many New York students trapped in failing schools? Math teacher Gary Rubinstein tries to break down the data. Reposted with permission.
A loaded question that education reformers like to ponder is: Exactly how many students in the boroughs of New York City are trapped in failing schools?
In November 2014 there was a report put out by Families For Excellent Schools called ‘The Forgotten Fourth’ in which they put the number at 143,000 which is about 15%. This 143,000 number was quoted for months on the news and was featured in a New York Post op-ed by Eva Moskowitz, founder and CEO of the Success Academy charter school network.
Two weeks ago, July 2026 The New York Post had an article about a new report with an updated number of how many NYC students are trapped in failing schools. The new number is 409,379 or 43%. This is based on a newly released report called ‘By Any Honest Measure’ and written by, of all places, Success Academy.
So how, exactly, did the number of students trapped in failing schools triple in the last twelve years? It all depends on how the word ‘failing’ is defined.
But defining what is and what is not a ‘failing’ school is a very subjective thing and I’m going to examine in this post how ‘By Any Honest Measure’ defines it to arrive at the 409,379 number and how slight adjustments to their metric would result in a very different number.
In the 2014 paper, the measure of a failing school was a school in which fewer than 10% of the students passed the math state test and fewer than 10% of the students passed the ELA state test. Just looking at pass rates can be misleading because you could have a school, for example, in which 90% of the students get a 2.9 and the rest get over a 3.0 and that would be 10% ‘passing’ while you could have another school where everyone gets exactly a 3.1 and they have 100% passing even though the two schools are basically the same in terms of their test scores. So just looking at percent passing can give a misleading picture, it is the most crude way of measuring school quality. More sophisticated ‘growth’ measures at least try to capture the idea that a school can be doing a good job even though the students haven’t gotten to a certain level yet. But for that ‘The Forgotten Fourth’ that was they way they defined ‘failing’ despite the flaws in that simplified metric.
But this new report from Success Academy has concluded that the number of students in ‘failing’ schools has tripled in the past 12 years. So how is ‘failure’ defined in ‘By Any Honest Measure’? For this study, a failing school is one where under 50% of the students have passed either the math or the ELA state tests. So to compare the two ways: ‘The Forgotten Fourth’ needed both math and ELA to be under 10%. ‘By Any Honest Measure’ needed either math or ELA to be under 50%. So right there, we should expect the number of students in failing schools to skyrocket. Well I downloaded the report and had to do some number crunching from the New York Department Of Education database so I can look at the numbers more closely.
The first thing that came to mind for me was, why is the bar for ‘failing’ now under 50% passing when it used to be under 10% passing. The Success Academy report says that there has been grade inflation as the average state test scores have been increasing while the NAEP scores have been stagnant. This is actually a valid point (one of the few they make) but it still isn’t detailed enough to raise the failure bar from 10% to 50%. As an experiment, I recalculated these numbers to see what if the rule was to find schools where students got less than 40% on at least one of the two measures. Just changing that one number from 50% to 40% lowered the number of students attending ‘failing’ schools 242,822 (if you notice that my 50% on the chart is a little off from the Success Academy calculation, there are some very small differences in the counting but the difference is negligible). And if we lower to 10% like in ‘The Forgotten Fourth’ now it is just 14,000 students instead of the 143,000. But I will acknowledge that cut off scores and what qualifies as getting a 3 on a test is its own political issue so it is hard to know what is considered an alarming number of students not reaching an arbitrary cutoff. This is more the reason to use a more sophisticated way of trying to measure school quality and not just saying 50% is the ideal threshold to measure whether a school is failing. Here is a chart showing how the Success Academy numbers would change had they used a different threshold.
The next big difference between the new ‘By Any Honest Measure’ report and ‘The Forgotten Fourth’ report is that for the older one, for a school to be failing, it had to be less than 10% for both measures while for this newer report, for a school to be failing, it had to be less than 50% for either measure.
So this means that if an elementary school had 49% of their students get a 3 on the math state test and 100% of their students get a 3 on the ELA state test, by the methodology in this report, that would be a failing school. That school would be failing while a school that got 51% on both tests would not be failing. This is really the critical flaw in this report so I decided to crunch the numbers to see how this one thing, the AND vs OR decision, affected the numbers.
First I wanted to check to see if this was a common thing, a school having fewer than 50% on one test and much greater than 50% on the other test. So I went to the Success Academy report where they showed the numbers they used and I found something very interesting.
So in their list of failing schools, they only report the number for the one that is under 50%. So what I did was download the entire NY State database and fill in every missing value. So, for instance, that first school on the list there, PS 165, I found that their ELA score was 57% So they had 48% Math and 57% ELA but because one of those numbers was less than 50% all 459 students in that school, including all the K-2 who didn’t take the test are part of the 408,000 students ‘trapped’ in the ‘failing’ school. Had a few more students in 3-6 gotten one or two more questions right on that math test, 459 students would have been spared from this trapping. Well, I ran two calculations. First I looked at all 905 schools on the failing list to see if there were a lot of schools like this. I found that about half the schools had gotten under 50% on just one test. So I added together the two numbers and found 173 that had a combined sum of over 100, like a school that had 48% on one and 57% on the other, PS 165. Here are some of those results:
Like Brooklyn Bridge academy, that ‘failing’ school that had 48.5% passing math and 97% passing ELA. Above I listed the the top 25 schools, but for the 173 schools that had a combined over 100, their total enrollment 117,385. This, right there, the unfair labelling of a school as failing despite having an average of 50% or higher comprises about 1/3 of the students the report contends are trapped in a failing school.
I think that even if a school had 35% math and 60% ELA so it doesn’t add up to over 100 but I still think it is unfair to label that school as a failure. If you only count schools where both scores are under 50%, the number of students enrolled in failing schools drops from 400,000 to 180,000. That’s how much can change when you change an ‘OR’ into an ‘AND.’ Whether Success Academy purposely left off the relevant other numbers from their spreadsheets so nobody would question if a school with 48.5% math and 97% ELA is truly ‘failing’ I don’t know, but I was glad that I have the technical know-how to fill in those missing numbers.
Looking at the chart published in The New York Post, one thing that jumped out at me was that they claim that 62% of high school students attend ‘failing’ schools while just 34% of elementary schools do. So even though high school is just four grades, 9, 10, 11, 12, they comprise almost 40% of the total students in the ‘failing’ schools.
So elementary schools have grades 3-6 state tests in math and ELA but high schools don’t. High schools, however, take a bunch of standardized state final exams called The Regents. One of those Regents is Algebra I and another one is ELA. Many students take Algebra I in 8th grade so only the lower performing students take the Algebra I Regents in high school and those scores are skewed. For elementary schools where four different grades are used to make their math score, trying to capture the quality of a high school by just looking at the Algebra I scores for students who came in not already having taken Algebra I in 8th grade is not going to be a fair way to judge the school, especially since getting less than 50% on Algebra I, regardless of how the school did on ELA Regents, dooms the school as failing and the entire enrollment to be trapped in it.
And since a school is counted as failing if fewer than 50% pass either the Algebra I or the ELA Regents, the entire high school is labelled failing based on this one test. So the first analysis I did on the high schools was to look at the 297 schools and see how many had gotten less than 50% on both which ended up as 74 schools. Of the rest, 210 had gotten under 50% on Algebra I but more than 50% on ELA while only 9 had gotten under 50% on ELA but over 50% on Algebra I.
One of the 9 entries in the category of over 50% Algebra 1 Regents and under 50% ELA regents is the ‘English Language Learners – Inter Supp’ isn’t actually a school with 276 students trapped but a category that happened to be in the database and which Success Academy included as 276 students to add to their list.
As a last irony, I looked at Success Academy’s Algebra I Regents scores. To someone who doesn’t know about the infamous ‘curve’ on this test, the 94% pass rate might seem fine. But only 52% of their students got a level 4 or level 5. The other 48% of their students got either levels 2 or 3. To get a level 4, you have to get 51 points out of 82, which is a 62% (which gets ‘curved’ to a 75). Level 2, though technically passing, is curved up from as low as 35% correct to 51% correct. So 48% of Success Academy, had New York State not done this generous curve, would have gotten less than a 62% on the Regents which would be failing. And, based on a bell curve, it is likely that a few of their level 4 students would have gotten under a 65. So by a pretty reasonable alternative way of scaling the Algebra I Regents, fewer than 50% of Success Academy students would have passed it and, by their way of measuring high schools, Success Academy Harlem I would be a failing school in which all the students there are trapped.
OK, I think you get the point. ‘By Any Honest Measure’ is anything but. It is really kind of embarrassing that a company with tens of millions of dollars would probably pay about 50 grand to create this slop. Sadly, this report has been picked up by the media and it will surely be quoted for years to come.