Akil Bello: The Reckless Rankings Game
In the Chronicle of Higher Education, testing guru Akil Bello looks at the racket that is the U.S. News college rankings system, looking at its recent abuses as well as its troubled history.
Outcomes-based rankings, in particular, have a troubled history. They are largely founded on the work of James McKeen Cattell, a psychologist — and eugenicist — at Columbia University, and Kendrick Charles Babcock, a specialist in the Bureau of Education, a precursor of the U.S. Department of Education. In 1903 Cattell created an evaluation of colleges based on how many “eminent men” were producing work on their campuses, and he used those results to devise a ranking. He also believed that the West was in decline, and that we could “improve the stock by eliminating the unfit or by favoring the endowed.”
While Cattell was sounding the alarm about the decline of “great men of science,” the Association of American Universities asked Babcock to determine which colleges best prepared their students for graduate school. The AAU believed that by working with the impartial Bureau of Education, the rankings would gain greater acceptance. However, an early draft of Babcock’s report leaked, and the ensuing backlash from lower-ranked colleges caused the sitting president, William Howard Taft, to issue an executive order to quash the report.
U.S. News took things in a slightly different direction.
U.S. News editors, by contrast, chose to base the first version of their rankings entirely on a reputational survey of 1,300 college presidents, many of whom had no familiarity with the institutions they were rating. The pool of raters eventually expanded, but problems remained. A National Opinion Research Center report commissioned by the magazine in 1997 found that, for raters, putting institutions into quartiles was “an almost impossible cognitive task.” The center also pointed out that each rater had been asked to rate a huge number of institutions — about 2,000.
The end result has been a system just screaming to be gamed run by an organization desperate to hold onto its clout in the marketplace.
Colleges will continue to engage in deception, manipulation, influence-wielding, and outright lying to shift the rankings. The rankings bring out the worst in colleges and harm the higher-education landscape. Public colleges suffer because the rankings are tilted to favor wealthier, smaller, private colleges. Colleges that choose to focus on excelling in areas outside of U.S. News’s criteria, or that don’t want to participate in the rankings at all, suffer because uninformed students decide where to enroll based partly on rankings.
And it should be noted that there is no reason to think that their rankings of high schools are any better.
Read the full article here. The Reckless Rankings Game (chronicle.com)