Rejecting University Rankings: Throwing the Baby Out With the Bath Water
This week Berend van der Kolk published a call to ban university rankings. He concludes: ”So, let’s have (inter)national and/or local discussions among academics about what sorts of places we want our universities to be, without interference from commercial rankers. Let’s replace the frame of academia as a competition with a frame that recognizes and rewards cooperation. Let’s put an end to the ranking circus.”
While I share with Van der Kolk that the rankings should not reflect the ultimate goal of the university, a high position does provide an important indication of the extent to which the institution in question is able to perform at the (world) top level.
Van der Kolk proposes three reasons why rankings are defunct. One is that the comparisons boil down to equaling apples with oranges, which, additionally, impedes cooperation between universities as one university can only become better at the expense of another! His second objection is in the subjectivity of the selected metrics, while the third objection is that the metrics incite universities to compete on performance in these metrics while “rankings favor uniformity over uniqueness. For instance, rankings often fail to appreciate local differences and small-scale, meaningful connections to local communities.” Van der Kolk coins the latter as commercialism.
The comparison argument does not hold as long as rankings are based on objective metrics. A university whose academics are able to publish in The American Economic Review (economics faculty) or The Lancet (medicine faculty), are clearly performing better than universities whose academics (here faculty in economics and medicine) are unable to publish in journals of lower rank. Lower rank is then measured by the quality of the review boards that review the work; for each research field the we tend to know who is excellent and this is reflected in the quality of the academics that are (not) invited for editorial boards. Indeed this does lead to competition between institutions but does not refrain academics from cooperating. Most published papers containing joint work are written by academics from different institutions.
Having said this, I do agree that one should be cautious in counting the number of publications per academic. It is inconceivable -in my field of accounting- that one academic publishes more than five papers a year in top journals with her/his colleagues. Such a record reflects either a low contribution to the paper or worse.
His second objection is about the subjectivity of the metrics. I guess when it come to publishing in the best journals the selection of journals is what would be referred to as intersubjective where the conventional concept of truth is suggested by Henri Poincaré (see Vladimir Vittikh, 2015) where the truth is conceived “as the result of a convention, an agreement which may be achieved to recognize certain subjective knowledge to be true for a restricted circle of actors.” The restricted circle here would be the academics who cover a particular field of research. Where is than the subjectivity in selecting the American economic review as a top journal that counts in the ranking? What would stop a group of other academics in -for instance- the field to form an alternative group of researchers to replace the current intersubjective agreement?
The third objection is that the rankings lead to too much conformity since everyone is going for the same topics that underly these rankings. Well when it comes to top journals I see no objection in that. What is the use of conducting second-rate research?
What I do miss in these rankings is the impact on society. This is -for instance- impounded in the English system where the university must present cases to demonstrate that the research insights are uses in practice. It would be good if universities are able to make that case.
While rankings are not the ideal, the alternative of no ranking is even less attractive. Imagine if we abandoned the criterion of top-level research, what would be the consequence? Currently, every university employee knows they must convince top-level people in their field of the importance of their work. After all, before publication, an assessment committee reviews the researcher’s paper, and if it is technically sound and constitutes a new contribution, publication follows. Otherwise, it is rejected. These two criteria are useful because they motivate the researcher involved to come up with something truly new and employ the most appropriate analytical techniques. Without these criteria, there would be no incentive to produce innovative work.
By abandoning rankings, the authors fall into the Nirvana Fallacy, discarding good things because they are imperfect. However, we won’t reach heaven in our lifetime, but we must strive for it, and you don’t do that by discarding good things because they don’t meet the heavenly criteria! Of course academics must produce the best works possible in their field, and the way to measure this is in the extent they can convince the best people in their field!
While I share with them that the rankings do not reflect the ultimate goal of the university, a high position does provide an important indication of the extent to which the institution in question is able to perform at the (world) top level.
Reference
Vittikh, Vladimir A. “Introduction to the theory of intersubjective management.” Group Decision and Negotiation 24.1 (2015): 67-95.
