Dear Readers,
U.S. News & World Report has published its annual rankings of universities and colleges, and as expected, the annual media fanfare has followed. I published this piece seven months ago after news surfaced about the persistent flaws of U.S. News's ranking system. It serves to remind us to be prudent when relying on data to make decisions and to ground any data gathering and findings on a solid ethical foundation.
U.S. News & World Report's college rankings have come under increased scrutiny in recent years as a handful of colleges have admitted to falsifying results or leaving out pertinent admissions test scores. The latest culprits: Bucknell University in Pennsylvania and Tulane University's business school in New Orleans. They follow George Washington University, Claremont McKenna College in California, and Emory University in Atlanta.
This week’s article in the Washington Post entitled “Five colleges misreported data to U.S. News, raising concerns about rankings, reputation” that describes the problem was disturbing on several levels.
For starters, knowing that the schools' data are not independently verified inevitably raises a key question: What other schools have misreported data over the years to score a higher ranking?
In addition, the link to Monday's U.S. News blog shows that the magazine doesn't grasp the crux of the problem. Robert Morse, its higher education analyst, wrote: “The difference between Bucknell's misreported data and newly reported data wasn't significant enough to affect the school's numerical rank. Therefore, based on our calculations, Bucknell's published numerical rank is correct and will not change."
Let's set statistical technicalities aside. Bucknell disclosed that it deliberately omitted some students' SAT scores for the past seven years. That significant ethical lapse should be enough to strip its ranking.
In further evidence of the magazine's blinders, its editor, Brian Kelly, is quoted in the article saying, “These are institutions that teach ethics. If they can’t keep their own house in order, they’ve got a problem. It’s their problem, not my problem."
Not really. Universities do not merely teach ethics. They are rooted in ethics as truth-seeking institutions. But so is the magazine that employs Kelly. And such institutions designed to serve the public good are derelict in their primary responsibility every time ethical violations occur. That makes it Kelly's problem as the leader of U.S. News. It is no wonder why Americans’ confidence in their institutions is in the cellar. The only three institutions that garner high ratings are the military (75 percent), small business (63 percent), and the police (56 percent), according to a Gallup Poll taken last year.
For better or worse, U.S. News's college rankings have been informing people’s decision making for about 30 years now. I used the rankings when I was considering which college to attend and then when I was looking at graduate schools in public service. Of course, I looked at other measures, too, and visited the institutions to survey the scene and talk to students and faculty. But there were certain periods of time when my friends and I could just about recite the rankings without even looking.
For all of us, the disclosures are just more reminders that we need to tread carefully when relying on data to guide our thinking and our decisions, especially when human malfeasance and computer bugs are such common occurrences.
This is especially true given the move toward using big data to understand patterns in a variety of disciplines, and education is no exception. Indeed, people are accustomed to throwing around the slogan coined by statistician W. Edwards Deming, “In God we trust, all others must bring data.”
But the God reference hints at something greater: that data must fit within a solid principled framework to lead to any valid conclusion.
That's why schools must continue to inculcate principles such as virtue, honor, and integrity in students and regularly reward such behaviors. Schools must reinforce ethics among administration, faculty, and staff as well.
For its part, U.S. News ought to make time to get its own house in order by suspending its college rankings. It would send a clear message that it values integrity as it says it does and affirm that data is no substitute for ethics. That would truly be a public service.
Please see NAIS’s Ranking Statement here: http://www.nais.org/Articles/Pages/NAIS-Statement-3a-On-Ranking-Schools-145361.aspx
The views expressed here reflect those of the author alone and do not necessarily reflect the views of NAIS. Please contact the author, Ari Pinkus, at [email protected] with comments and suggestions for future blog posts.