Does Research Being in a Review Article Cannibalize Your Citations?

Scientific knowledge is ever-expanding. As new ideas and discoveries are created, it is increasingly difficult for researchers to keep track of everything being discovered and published. Scientists often turn to summaries to make sense of it all. Review articles are a common means of curating the knowledge amassed on a topic.

On the one hand, it is reasonable to think that scientific review articles will bring welcome attention to the research they summarize. After all, if a review’s main purpose is to identify the central works in a field—to separate the wheat from the chaff—then the articles they highlight ought to become more visible and more highly esteemed as a result. On the other hand, some argue just the opposite: scientists may cite the review itself in place of the articles reviewed, and reviews may therefore ‘poach’ attention away from the very research they are supposed to highlight.

It is fair to ask, then: do reviews promote the articles they cite, or do they poach the attention those articles would otherwise have received? In a recent article, co-authored with Daniel A. McFarland and published in the American Sociological Review, we use a corpus of nearly six million scholarly articles to show that this seemingly simple question has a surprisingly complex answer. Yes, reviews do, on average, draw attention away from the articles they cite. But they do so selectively, boosting the popularity of a small selection of the articles they cite. In fact, the way that reviews guide scientists’ attention may be a central part of their role in the production of scientific knowledge more generally.

Reviews have a substantial effect on the future reception of the articles they cite. By looking at articles cited in Annual Review journals between 1990 and 2016, we find that almost 70 percent can expect to be cited less than they would otherwise. Moreover, the size of this effect can be quite dramatic: the median cited article will have 38 percent fewer citations over the remainder of its scientific lifetime after being included in just one review article. This finding alone seems to provide good evidence of the ‘poaching’ hypothesis mentioned above. But, digging deeper we find that the story is a more nuanced. While most cited articles will receive less attention going forward, this effect is far from universal. A small minority of cited articles experience a considerable increase in future citations. This suggests that there is something more interesting going on than a process of diverting citations from original research to review articles.

The loss of attention for the majority of articles is due to review articles’ restructuring perceptions of emerging scientific domains. Reviews are more than just lists of relevant references—they create a narrative, tying existing work together to make a complex domain more intelligible. Reviews aim to create a portable version of an area of scientific research, or a “packed down” version of the major findings that will be useful to a wide audience. In this respect, reviews are not just summaries, but translations of scientific subfields. Scholars of science and technology studies have long argued that translation like this is not a neutral act. In order to make a body of knowledge accessible to outsiders, it needs to be reconfigured in a way that is intelligible to people with different expertise and different experience.

By comparing review articles published in 54 Annual Review journals to articles found in the rest of the WoS corpus, we find that Annual Reviews perform this translation in predictable ways that restructure and refocus scientific domains. To analyze a research area’s structure we construct co-citation networks on each set of cited works. We look at the changing structure of these co-citation networks before and after their inclusion in a review.

We find that reviews are most often written about newly emerging scientific subfields, and those subfields tend to be made up of several tight research clusters that are only loosely linked to one another. And those links between clusters are created by a handful of ‘bridging’ articles which tie disparate research efforts together (see figure below).

The translation that reviews perform results in a radical restructuring of these domains, transforming the co-citation networks from clustered communities into highly centralized bridging structures. Reviews create representations of emerging subfields that are focused a small set of bridging articles that become the canonical citations for future work in that area. These publications become exemplars of the area of scientific research and enjoy considerably more attention going forward than if the review had not been published.

Overall, the research shows that reviews play a systematic and, we argue, important role the production of scientific knowledge. Reviews do detract from the prominence of most of the articles they cite. But by doing so they promote the works that most efficiently characterize the work being performed in and across clusters of research, thereby preserving the bridges that speak to larger body of work. Emerging areas of scientific knowledge are complex and difficult to represent as a coherent whole. By reducing the literature in an area into an easy-to-swallow bite, reviews help establish the coherence of that area and represent it as a unified whole. At the same time, the curation that reviews perform is opinionated—not every review author would center the same set of exemplary work. Our results underscore the potential for reviews to reinforce existing barriers to access in scientific research based on, e.g., gender or race. By simplifying emerging subfields as centralized, cohesive groups of work, reviews shape those subfields through the eyes of their authors in a way that allows them to become established as legitimate fields integrated into the body of scientific knowledge. As the body of knowledge continues to expand, one finds it hard to imagine how we can make sense of it without the curation provided by reviews.

This post draws on the the article “Creative Destruction: The Structural Consequences of Scientific Curation,” by the author Peter McMahan and Stanford University’s Daniel A. McFarland, published in American Sociological Review.