David Brooks can sure sound as if he knows what he’s talking about. But a recent piece about education turns out to be full of baloney, and Mark Weber has the data skills to point out the problems. For instance, when it comes to Brooks’ assertion that states have become slacker and slacker in their meaurement of proficiency:
Under federal law and since NCLB was enacted in 2001, all states have had to test their students in Grades 3 through 8 in math and reading. The tests, however, vary from state to state, so there is no common scale on which to assess average student performance. However, the National Center for Education Statistics “maps” the results of state tests on to the “gold standard” national test: the National Assessment of Educational Progress (NAEP).
In the simplest terms: NCES looks at the percentage of students a state deems “proficient” on their own test. They then look at the NAEP data and determine what that test’s score is for the same percentage of students who took the NAEP. That allows us to look at the differences in proficiency between states: which states set the bar at a relatively high NAEP score, and which set the bar lower.
Because NCES does this every time the NAEP is administered, it also allows us to assess how standards have changed over time. I took this data and averaged it across all states for each year of the NAEP (weighted by student enrollment). There are some cautions involved in this: for one, state tests might emphasize different content at different times, so increases in mapped scores might not reflect more “rigor.” We’re also assuming the NAEP is a consistent measure of student learning across time (a pretty reasonable assumption, IMO). But I’d argue this is our best estimate of whether state standards have, on average, moved higher. Have they?
Over the last two decades, the definition of “proficiency” on state tests has, on average, become more rigorous, not less.
Read the full post for more explanation of what Brooks gets wrong.