Research

NIH: Exploring the Publication Gap Between Social/Behavioral and Biomedical Research

February 19, 2021 2669
forested and deforested hillside

[William T. Riley is the director of the National Institutes of Health’s Office of Behavioral and Social Science Research. The article in PLOS One was co-written with Katrina Bibb, Sara Hargrave and Paula Fearon.]

In November 2020, we published an article in PLOS ONE on “Publication rates from biomedical and behavioral and social science R01s funded by the National Institutes of Health.” In that article, we reported that the percent of R01s – a standard grant mechanism in NIH — with zero publications five years from the project start date was low overall (2.4 percent) but higher for behavioral and social sciences research (BSSR) grants (4.6 percent) than for non-BSSR grants (1.9 percent). We also reported that the time to first publication was slower for BSSR than for non-BSSR grants. Differences remained even after controlling for factors such as conducting human research or clinical research. 

We did not generate this research question about publication differences between BSSR and non-BSSR R01 grantees de novo. NIH has been concerned about publication productivity, particularly for clinical trials, for some time, and the NIH clinical trials policies were developed in part to ensure that results reporting occurred, regardless of null results or publication bias. More recently, NIH is performing an ongoing quality review of its Center for Scientific Review study sections. This review, called ENQUIRE, includes, among the metrics considered, the productivity and impact of the research published from the grants reviewed and funded by each study section, specifically the number of grants within a study section that fail to produce a single publication in a timely manner.

Early analyses of some of these study sections showed a greater number of grants with zero publications among study sections that review mostly BSSR grant applications relative to other study sections. We hypothesized at the time that BSSR grantees would understandably take longer to publish given the large proportion of human research and longitudinal clinical or applied research relative to our biomedical colleagues. That hypothesis, however, begged for research to test this hypothesis, leading us to analyze the publication productivity of grants funded by the NIH from 2008 through 2014.

We chose to focus specifically on R01 grants, the most common grant mechanism of the NIH, because, given the preliminary data required and the level of investment provided by this award, the likelihood of failing to produce a single publication should be quite low. As expected, the rate of zero publications from a grant within five years from the project start date was low (2.4 percent). This rate is much lower than found in prior clinical trials publication research, but clinical trials research, by its human research and longitudinal nature expected to take longer to publish, and the outcome variable in most of these prior studies with the primary outcome paper, not any paper associated with the grant as it was in this analysis. While we do not know what the optimal rate of zero publications within this time period should be, for the $3 million to $4 million investment of the typical R01, it is reasonable for the taxpayers and their representatives to question why any R01 grant does not produce a single scientific publication within five years of funding. 

Of course, there are many legitimate reasons why this might be the case, and why the rate of zero publications and the lag to first publication is slower for BSSR vs. non-BSSR grants. These include:

  • Some research does not produce publishable findings. Research plans do not usually go as planned. Risky research may not be feasible, and publication bias against null findings remains despite efforts to reduce such bias. Yet even “failed” projects provide useful insights for the field, hence the NIH efforts toward required registration and results reporting. 
  • Some research takes longer than others to complete and publish. Research involving humans takes longer than research involving cells, and longitudinal studies take longer than cross-sectional studies. Our analyses could account for only a few of these factors (e.g., clinical trial or not, human research or not, child research or not) due the constraints of the data available in the NIH grant database, and these factors were shown to be important contributors to the time lag to first publication, but they did not fully explain the difference in publication lag times between BSSR and non-BSSR grants.
  • Scientific disciplines have different publication “cultures”. Economics and social sciences tend to publish less frequently but with lengthier publications than their biomedical colleagues. Computer scientists place a higher value on scientific communication via conference presentations than via journal publications. 
  • Pubmed Central, the full-text archive operated by the NIH’s National Library of Medicine, may not be an accurate source of publications associated with a grant. Biomedical journals routinely ensure that their publications, and their associated grant funding information, are included in Pubmed Central, but is unclear if social and behavioral science journals, especially ones without a health-relevant scope, do so routinely. Since NIH relies on Pubmed Central for evaluating the publishing productivity of the grants it funds, it is incumbent on social and behavioral science journals and the authors of articles published in these journals to ensure that the publication record is included in Pubmed Central and that the grants that supported the work described in the publication are appropriately associated with the grants. 

There are many potential reasons for the findings from this study regarding publication lag times, and we plan on reanalyzing the publications from these grants in a couple of years to see if with more time, more research from these grants was published. But the take-home message of this study is while there are many factors that contribute to longer times to publication and failure to publish in a few situations, it is the responsibility and obligation of NIH-funded investigators to the research field, to the taxpayers who support their research, and to their study participants to publish or make public in some form the results of their research in a timely manner. 


This post was originally titled, “Slower Publication Rates from Social and Behavioral Researchers Compared to the Biomedical Researchers Supported by NIH.”

William 'Bill' Riley was appointed National Institutes of Health associate director for behavioral and social sciences research, and director of the NIH Office of Behavioral and Social Sciences Research in August 2015. A clinical psychologist, he also has an appointment as professorial lecturer in the School of Public Health at George Washington University. His research interests include behavioral assessment, psychosocial health risk factors, tobacco use/cessation, and the application of technology to preventive health behaviors and chronic disease management.

View all posts by William Riley

Related Articles

Three Decades of Rural Health Research and a Bumper Crop of Insights from South Africa
Impact
March 27, 2024

Three Decades of Rural Health Research and a Bumper Crop of Insights from South Africa

Read Now
Using Translational Research as a Model for Long-Term Impact
Impact
March 21, 2024

Using Translational Research as a Model for Long-Term Impact

Read Now
Coping with Institutional Complexity and Voids: An Organization Design Perspective for Transnational Interorganizational Projects
Research
March 19, 2024

Coping with Institutional Complexity and Voids: An Organization Design Perspective for Transnational Interorganizational Projects

Read Now
New Funding Opportunity for Criminal and Juvenile Justice Doctoral Researchers
Infrastructure
March 7, 2024

New Funding Opportunity for Criminal and Juvenile Justice Doctoral Researchers

Read Now
Economist Kaye Husbands Fealing to Lead NSF’s Social Science Directorate

Economist Kaye Husbands Fealing to Lead NSF’s Social Science Directorate

Kaye Husbands Fealing, an economist who has done pioneering work in the “science of broadening participation,” has been named the new leader of the U.S. National Science Foundation’s Directorate for Social, Behavioral and Economic Sciences.

Read Now
Did the Mainstream Make the Far-Right Mainstream?

Did the Mainstream Make the Far-Right Mainstream?

The processes of mainstreaming and normalization of far-right politics have much to do with the mainstream itself, if not more than with the far right.

Read Now
The Importance of Using Proper Research Citations to Encourage Trustworthy News Reporting

The Importance of Using Proper Research Citations to Encourage Trustworthy News Reporting

Based on a study of how research is cited in national and local media sources, Andy Tattersall shows how research is often poorly represented in the media and suggests better community standards around linking to original research could improve trust in mainstream media.

Read Now
5 1 vote
Article Rating
Subscribe
Notify of
guest

This site uses Akismet to reduce spam. Learn how your comment data is processed.

0 Comments
Inline Feedbacks
View all comments