Industry

Don’t Mistake Cruelty for Rigor in Peer Review

June 27, 2022 835
The Conversation logo
This article by Catherine Clase, Josee Bouchard, Manish M Sood, Rachel Holden and Sunny Hartwig originally appeared on The Conversation, a Social Science Space partner site, under the title “Peer review: Can this critical step in the publication of science research be kinder?”

Democracy has been called the least worst system of government. Peer review is the least worst system for assessing the merit of scientific work.

Peer review is the written evaluation of a paper by other experts in the field. Though this sounds like assessment by equals, the power imbalance created by the roles of reviewer and reviewed distorts the relationship and affects the tone of the review. Reviews can be patronizing, demanding and unkind.

It is painful to read harshly worded criticism of work that has taken a team hundreds or thousands of hours and been submitted hopefully and in good faith. From our experience, we know that reviews can be accurate, robust and make every scientific point while using language and tone that is helpful and supportive.

Supportive review

We are a team of editors of an open-access Canadian kidney journal, the Canadian Journal of Kidney Health and Disease. When we founded our journal in 2014, supportive review was the first of our guiding principles. Since then, we have written supportively as editors, selected reviewers who write supportively and participated in training the next generation of Canadian kidney scientists to conduct reviews that are complete, rigorous and kind.

Supported by a larger group of like-minded people from multiple disciplines, we recently published an editorial outlining these principles. A dozen other kidney journals expressed their support for the idea, with Nature Reviews Nephrology, NDT and Pediatric Nephrology publishing co-ordinated editorials recommitting to principles of constructive criticism.

The long process of research

Scientific papers condense a large amount of work into a structured format, usually no longer than four to eight times the length of this article. The work of a paper starts with an idea that may be developed by the team for a year or more before it crystallizes into an application for funding, which may go through rounds of revisions.

Once funded, people and budgets are assigned to the project and the work proceeds. The work can involve the time of multiple team members for months and even years.

When the work is complete, they write a paper, detailing what they did, how and why, what they found and what they think it means. This paper itself is often the product of hundreds of hours of work, with multiple authors contributing their specific expertise and working on the messaging of the whole.

The journal receives the manuscript and assigns an editor, who assigns peer reviewers. Peer reviewers are other scientists working on similar topics. They must be totally unconnected with the people writing the paper. With notable exceptions, most journals employ single-masked peer review: the reviewer sees the authorship of the paper but the authors of the paper will not see who wrote the review.

Peer reviewers are not paid or rewarded for their review of the manuscript — they take it on as part of the work of academic life. Essentially, it is an unrewarded activity performed by people who are themselves authors. It varies by discipline, but in biomedicine, they may spend three to six hours on a review.

Harsh reviews

How does this altruistic activity, undertaken by a reviewer who is very familiar with the author role, lead to such pain and frustration for other authors?

We think that scientists sometimes confuse harshness with intellectual rigor and that a reviewer’s experience of harshness in reviews of their own work, amplified by the power imbalance between reviewer and reviewed, leads to perpetuation of harsh and unhelpful review. Other reviewers and editors avoid these pitfalls entirely.

“It looks to me like one of your first attempts at scientific publishing, and I can understand that you are also writing in a non-native language,” wrote one anonymous reviewer to a mid-career woman scientist with 13 first-author peer-reviewed publications. “I just want to give up today,” she wrote.

But she won’t. Scientists are prepared to receive this kind of feedback and be hurt over and over in the name of science. As editors, we believe there is a better way — that feedback should be rigorous, but will be more readily incorporated if kindly given, to the advancement of science.

These are not new ideas. In 2006, professor Mohan Dutta suggested 10 commandments for reviewers, all of which focus on the collaborative nature of relationship between reviewer and reviewed. Advice for reviewers often includes a recommendation to write constructively, though sometimes this is phrased as something like “write constructively, and then turn to criticism,” as if those are mutually exclusive.

We can take this principle further and — thanks to our community of reviewers in kidney medicine — we and other kidney journals make a commitment to kindness in review. Dutta’s 10th commandment is “do unto others as you would have them do unto you.” Every branch of science would be improved by implementing this idea.

Catherine Clase is a professor of medicine in the division of nephrology, and an associate member of the department of health research methods, evidence and impact at McMaster University. Josee Bouchard is a neprologist and professor of medicine at the Université de Montréal. Manish M Sood is a physician and professor of medicine at L’Université d’Ottawa/University of Ottawa. Rachel Holden is a nephrologist and a professor in the Department of Medicine with a cross-appointment to the Department of Biomedical and Molecular Sciences at Queen's University, Ontario. Sunny Hartwig is an associate professor at the University of Prince Edward Island.

View all posts by Catherine Clase, Josee Bouchard, Manish M Sood, Rachel Holden and Sunny Hartwig

Related Articles

Opportunity to Participate in RFI on Proposed National Secure Data Service
Announcements
May 28, 2024

Opportunity to Participate in RFI on Proposed National Secure Data Service

Read Now
Why Social Science? Because It Can Help Contribute to AI That Benefits Society
Industry
May 28, 2024

Why Social Science? Because It Can Help Contribute to AI That Benefits Society

Read Now
Biden Administration Releases ‘Blueprint’ For Using Social and Behavioral Science in Policy
News
May 17, 2024

Biden Administration Releases ‘Blueprint’ For Using Social and Behavioral Science in Policy

Read Now
Celebrating 20 Years of an Afrocentric Small Scholarly Press
Communication
May 7, 2024

Celebrating 20 Years of an Afrocentric Small Scholarly Press

Read Now
Why Social Science? Because It Makes an Outsized Impact on Policy

Why Social Science? Because It Makes an Outsized Impact on Policy

Euan Adie, founder of Altmetric and Overton and currently Overton’s managing director, answers questions about the outsized impact that SBS makes on policy and his work creating tools to connect the scholarly and policy worlds.

Read Now
The Importance of Using Proper Research Citations to Encourage Trustworthy News Reporting

The Importance of Using Proper Research Citations to Encourage Trustworthy News Reporting

Based on a study of how research is cited in national and local media sources, Andy Tattersall shows how research is often poorly represented in the media and suggests better community standards around linking to original research could improve trust in mainstream media.

Read Now
A Behavioral Scientist’s Take on the Dangers of Self-Censorship in Science

A Behavioral Scientist’s Take on the Dangers of Self-Censorship in Science

The word censorship might bring to mind authoritarian regimes, book-banning, and restrictions on a free press, but Cory Clark, a behavioral scientist at […]

Read Now
0 0 votes
Article Rating
Subscribe
Notify of
guest

This site uses Akismet to reduce spam. Learn how your comment data is processed.

0 Comments
Inline Feedbacks
View all comments