Young Scholars Can’t Take the Field in Game of Academic Metrics

Academic publishing is a game. And a lucrative one for those who win. There is ample evidence of excessive self-citation; citation cartels, researchers trading citations amongst themselves; coercive citations, journal editors pressuring researchers to cite their own journal to boost its ranking; and now a citation black market, where 50 citations can be purchased for as little as US$300.

The same investigation that uncovered this citation black market also found that Google Scholar is easily manipulated. That a researcher’s profile can be boosted thanks to fraudulent citations and that Google Scholar is the primary destination to evaluate a researcher’s profile across the world’s top ten universities.

Beyond the obvious academic misconduct, this is problematic because publication metrics remain the easiest and most used way to evaluate a researcher’s contribution. It is far easier to quantify a researcher’s output than it is the actual outcome of their work.

This may be the first time an explicit citation black market has been uncovered, but the corruptibility of academic publications has been known for well over a decade. Researchers face being coerced to cite by editors attempting to inflate their own journal’s impact factor. Those who bow to the pressure of coercion are then more likely to have their paper published. And with intense competition to publish in high-ranking journals, they are encouraged to manipulate their own metrics, those of their colleagues and those of the prestigious journals in which they desire to be published.

This system works for those who work the system, but at the great detriment to the broader community. For many it has become a prerequisite for career progression. All of this isn’t to say that such misconduct is pervasive. But, it has been proven widespread enough to prompt serious thought on how researchers and the contributions of their work are evaluated, valued, and rewarded.

Incentives to game the system redirect efforts from more intangible and difficult to measure forms of communication, such as public outreach. It also diminishes the incentives for collaborating and communicating new findings with colleagues prior to publication. Finally it locks researchers into a vicious cycle of dependency, unable to advocate for the radical change needed to mitigate these issues.

Perhaps the only way to stop cheating is to remove the motivation. So, ultimately, there appears to be only one solution to prevent the gamification of academic metrics and these strategic plays for prestige. That is: a cultural shift in the kinds of scholarship that are acknowledged, supported and rewarded.

As part of my research, I have been speaking with a broad range of academics from Australia, New Zealand, the United States and the UK, who are expanding their scholarship outside these traditional circuits of prestige. What I have found is a strong desire for change and a sense that change is coming, even if the academe moves at a snail’s pace.

The academics driving this change aren’t lesser scholars, jealous of others’ high h-index. What unites them is a sense of purpose and where they seek meaning in their work. With their commitment to social impact, academics engage their research with broader audiences to fulfil its full potential. They derive satisfaction seeing the fruits of their intellectual labor out in society. Not behind the paywall.

DrHossai Gul makes a potent point:

To connect their research with audiences beyond their peers, academics are constantly innovating. That’s how academics have been finding success with creative research translation, like on YouTube and with podcasts.

These aren’t vanity projects. It’s the most effective way, academics believe, to connect their research with non-academic audience. And, according to associate professor Vitomir Kovanović, “We want to have engagement because that will lead to something else. So whether it’s more funding, money, reputation, whatever that might be.” Embracing creative modes of engagement also achieves institutions’ objectives.

Still, even though institutions applaud academics’ initiative, these endeavors aren’t usually acknowledged or rewarded in any meaningful way. Kovanović concedes, “Academics have almost no incentive.” Engagement metrics, both quantitative (such as views, likes, shares) and qualitative(quality and content of comments and conversations), currently mean little.

In principle, this work aligns with the impact agendas of modern universities. However, in the experience of the academics I speak with, impact-oriented activities, such as community outreach and engagement count for little when it comes to performance measures and career progression. So, academics do these things, against the status quo and at their own professional peril.

There is currently a heated debate around how academics should be assessed and where the appropriate balance lies between metrics and peer review in assessing the value of their contributions. However, these issues are systemic and deeply embedded in academic culture. The careers of most senior scholars have been built on publications, conferences, grants and positions of esteem within their academic discipline. For early- and mid-career academics, especially those experiencing precarious employment, not playing the game isn’t an option.

It is unlikely that change will be top down, institutions (with a few notable exceptions) have shown they aren’t ready or able to wean themselves off prestige metrics and adopt more meaningful measures of value and success. But, perhaps scaling small approaches is more promising. There are many academics, who have trailblazed new kinds of public scholarship, aimed for impact over self-interest, translated their work more creatively to wider audiences and have shaped public opinion to bring about a better world. Learning from their motivations and experiences can provide a blueprint for a more meaningful research system.