Higher Education Reform

Is Peer Review an Achilles Heel for Interdisciplinary Work?

August 16, 2016 1731

missing puzzle piece

Is competent peer review a missing piece in the puzzle of getting interdisciplinary proposals greenlighted?

The first empirical support for a long-standing complaint by interdisciplinary researchers was recently published in the leading journal Nature. The Australian National University’s Lindell Bromham, Russell Dinnage and Xia Hua showed that interdisciplinary research is less likely to be funded than discipline-based research proposals (“Interdisciplinary research has consistently lower funding success”).

LSE Impact logo

This article by Gabriele Bammer originally appeared on the LSE Impact of Social Sciences blog as “Why are interdisciplinary research proposals less likely to be funded? Lack of adequate peer review may be a factor” and is reposted under the Creative Commons license (CC BY 3.0).

They cleverly applied a technique from evolutionary biology that examines relatedness between biological lineages, using a hierarchical classification of research fields rather than an evolutionary tree. The relative representation of different field of research codes and their degree of difference were used as a proxy measure for interdisciplinarity. The results, based on five years of data from the Australian Research Council’s Discovery program, are robust and are unaffected when number of collaborators, primary research field and type of institution are taken into account.

What does it mean? Both main potential interpretations raise concerns and require action. The results may support the grumbling of interdisciplinary researchers about an unfair review processes, where reviewers with no interdisciplinary research experience hold sway over which interdisciplinary research is published. It is also possible that interdisciplinary research proposals are not as good as those which are discipline based.

What is required for adequate peer-review of interdisciplinary grant applications? I recently laid out the key issues in a paper for Palgrave Communications. This paper argued that only involving reviewers from the disciplines was likely to significantly short-change an interdisciplinary proposal.

First, the unknowns (expressed as research questions) that are addressed in interdisciplinary proposals may include issues of concern to disciplines, but are unlikely to be confined to those. Interdisciplinary research is also likely to address unknowns that worry stakeholders and unknowns that are central to the problem, both of which may well be outside the purview of disciplines. For example, in research I conducted on the feasibility of prescribing pharmaceutical heroin to dependent heroin users as a new treatment option, we addressed concerns raised by police, one of the major stakeholders, about the possibility that heroin prescription would cause a “honeypot effect”, an issue that was not on the radar of any of the disciplines involved. Similarly we addressed concerns about fostering more permissive attitudes to illicit drug use, which are integral to changing prohibitionist policies. Such issues may seem irrelevant to discipline-based reviewers and certainly cannot be adequately assessed by them.

Further, the discipline based unknowns that are relevant to the interdisciplinary research, such as estimating the number of heroin users (demography), assessing the ethics of heroin prescription (philosophy), and estimating the likely impact on the illicit drug market (economics) may seem pedestrian to discipline-based researchers, even though they are critical to assessing an interdisciplinary problem such as the feasibility of heroin prescription. Indeed in this project only the economists undertook research that was ground-breaking from a disciplinary perspective.

Discipline-based researchers are also ill-equipped to evaluate the integrative processes that an interdisciplinary proposal plans to use, such as assessing the systems view taken, the rationale by which disciplines and stakeholders were chosen for consideration in the study and how all the different insights will be combined. Along with the different unknowns addressed, integrative methods are what set interdisciplinary research apart.

These considerations are relevant to one kind of interdisciplinary research, where several diverse disciplines and stakeholders tackle a complex real-world problem. But there are also other kinds of interdisciplinary research, ranging from one person borrowing from other disciplines to tackle a problem in a new way (as Bromham and colleagues did in their paper), to teams working at the interface of two or more disciplines (which may lead to the development of a new discipline, as when quantum physics and biology combine to form quantum biology), to closely related disciplines converging to develop a new technology. Different peer review processes may well be required to do justice to these different kinds of interdisciplinarity.

At the moment these issues are in the funders’ too hard basket. And who can blame them? Apart from complaints about the inadequacy of current processes, the interdisciplinary community is too unorganized to suggest adequate ways forward. Key tasks include providing a usable definition of interdisciplinarity (that encompasses the main kinds), highlighting central issues for peer-review of each kind, and providing an adequate college of peers to undertake effective peer-review.

And what if the results by Bromham and colleagues result from the quality of the interdisciplinary research proposals submitted for funding not being of a high enough standard? That’s also the purview of the interdisciplinary community. Just as the disciplines police their own standards, interdisciplinarians need to make clear which parameters mean that interdisciplinary research is outstanding and which mean it is poor.

If we don’t organize, our research will always be judged by outsiders, who are less well equipped to do so than we ourselves are.


Related Articles

Universities Should Reimagine Governance Along Co-Operative Lines
Higher Education Reform
May 20, 2024

Universities Should Reimagine Governance Along Co-Operative Lines

Read Now
Striving for Linguistic Diversity in Scientific Research
Communication
May 1, 2024

Striving for Linguistic Diversity in Scientific Research

Read Now
Civilisation – and Some Discontents
Public Engagement
April 30, 2024

Civilisation – and Some Discontents

Read Now
Exploring ‘Lost Person Behavior’ and the Science of Search and Rescue
Featured
April 24, 2024

Exploring ‘Lost Person Behavior’ and the Science of Search and Rescue

Read Now
The Power of Fuzzy Expectations: Enhancing Equity in Australian Higher Education

The Power of Fuzzy Expectations: Enhancing Equity in Australian Higher Education

Having experienced firsthand the transformational power of education, the authors wanted to shed light on the contemporary challenges faced by regional and remote university students.

Read Now
Using Translational Research as a Model for Long-Term Impact

Using Translational Research as a Model for Long-Term Impact

Drawing on the findings of a workshop on making translational research design principles the norm for European research, Gabi Lombardo, Jonathan Deer, Anne-Charlotte Fauvel, Vicky Gardner and Lan Murdock discuss the characteristics of translational research, ways of supporting cross disciplinary collaboration, and the challenges and opportunities of adopting translational principles in the social sciences and humanities.

Read Now
Addressing the United Kingdom’s Lack of Black Scholars

Addressing the United Kingdom’s Lack of Black Scholars

In the UK, out of 164 university vice-chancellors, only two are Black. Professor David Mba was recently appointed as the first Black vice-chancellor […]

Read Now
0 0 votes
Article Rating
Subscribe
Notify of
guest

This site uses Akismet to reduce spam. Learn how your comment data is processed.

0 Comments
Inline Feedbacks
View all comments