Thomas Ultican: Testing Industry Scores Big in California
Yesterday the California State Board of Education decided to adopt a “student growth model,” which means that test scores will have to be pushed through some maths in order to spit out some numbers. And that means that somebody is going to make a bunch of money doing the pushing. Thomas Ultican has been digging into the details. Reposted with permission.
At 1:45 PM this Wednesday, the California State Board of Education (SBE) adopted a “student growth model” to evaluate student learning. It is a method fans of standardized test based accountability have been trumpeting. The big winner here is the testing giant Education Testing Services (ETS) who created the model to be used.
Board member Sue Burr who was appointed to the board by then Governor Jerry Brown made the motion for using the growth model. She carefully presented her motion directly from the state’s California Department of Education (CDE) staff report which recommended:
“The student growth model methodology, which includes using RG [residual growth] scores and the EBLP [Empirical Best Linear Prediction]hybrid approach to report aggregated student growth, and that the following score reporting be adopted:
“1. Report the EBLP weighted average for:
- Schools
- Student groups in a school
- The “All” student group in an County or District
- Student groups in a district with 500 or fewer students (with test scores)
“2. At the Local Education Agency level, report the simple average for all race/ethnicity and program participation student groups with more than 500 scores.”
Board member Patricia Ann Rucker seconded the motion. She is a legislative advocate for the California Teachers Association.
The measure was adopted by a unanimous 10-0 vote. The only member of the eleven-member board not voting for it was Board President Linda Darling-Hammond who was absent.
Developing an Accountability Model to Satisfy ESSA
In January 2016, the SBE discussed how to bring the state’s accountability vision in line with the federal Every Student Succeeds Act’s (ESSA) new accountability requirements. SBE received an information memorandum from the CDE stating,
“One such indicator would be a student-level growth model. This memorandum provides an overview of student-level growth models that can be used to communicate Smarter Balanced Summative Assessment results.”
In 2017, the state joined with testing vendor ETS to develop a Growth Model. After exploring several different models, the SBE selected the “residual gain”method for further study.
Again the state hired testing vendor ETS to evaluate the residual gain model in 2018. In a report, ETS stated that the model had low “cross-year stability.” In other words the results were not consistent. ETS developed an arithmetic manipulation method to solve the stability problem that they named the “Empirical Best Linear Prediction (EBLP).”
Student Growth Model Requires Opaque Math
The graphic above shows two types of growth models. One is the simple gain model which simply subtracts the third grade testing data from the fourth grade data to get a gain score. However, it is not really that simple. Mathematical manipulation is required to make the scale from the 3rd grade match the scale in 4th grade. This requires several assumptions that affect the results.
The method California has chosen adds another set of mathematical assumptions to create an expected outcome. That expected outcome is then subtracted from the 4th grade data to get the residual gain score. However since those scores are not stable, the ETS averaging scheme is being applied to make the results appear stable.
ETS is the Goliath of the standardized testing industry. It administers America’s most important standardized tests (including the SAT, PSAT, CLEP, AP, TOEFL, and Praxis exams) and develops many of them. ETS was organized as a non-profit in 1948. Today, their 2018 tax document shows this “non-profit” has revenues in excess of $1.5 billion. It also shows that 17 board members working 2-hours or less per week had an average income of $73,000 and 33 key employees had average incomes of over $533,000 per year with President Walter McDonald taking in $1,350,474.
The ETS “non-profit” is actually diving up a lot of profits.
Standards Based Education and Testing is Harming Kids
In December, Peter Greene put together a compendium of articles he has posted about the stupidity associated with standardized testing in schools. One particularly important article is about Daniel Koretz book the Testing Charade. This recognized testing expert from the Harvard Graduate School of Education gives us one more well-founded dismantling of the myth that standardized testing is anything other than destructive.
Student testing data does not indicate learning, teacher effectiveness or school quality and it is statistically noisy. A correlation study of how testing data is affected by various factors assigns r-values of between 1 and 0. A value of 1 means 100% correlated and a value of 0 means not correlated at all. When correlation studies are done with standardized testing data there is only one factor that has an r-value greater the 0.3 (weakly correlated) and that is family wealth which has an r-value of 0.9.
So if Burrito Canyon high school has excellent testing data that is always better than the scores at Pawn Valley high school, there is nothing known about the learning and teaching, but you can be confident students at Burrito Canyon have wealthier parents.
These growth models – that are now being used in almost every state in America – take in noisy testing data that has little valuable meaning and applies fancy arithmetic. It is the classic “garbage in – garbage out” situation.
Not only is standardized testing removing almost $2 billion dollars from America’s classrooms every year, it is doing damage to students. Standards based education is bad education theory because there are no standard children. Testing forces educators to teach to the test and it further narrows curriculum. Gifted education professionals have shown repeatedly that this methodology driven by corporations and billionaires is harmfully misguided.