I went to a talk at a conference a few years ago (remember those?) where the speaker referenced the ongoing debate around a paper by Ken Gergen entitled “Social Psychology as History.” The paper was published in the Journal of Personality and Social Psychology (JPSP), in 1973! The fact that a paper published nearly 50 years ago is still being debated by social psychologists today should remind us that the time horizons over which social science work has an impact is, on the whole, much longer than in STEM disciplines. And relatedly, social science scholars, within distinct ecosystems of reward and recognition, typically grow their reputations and expertise over a longer time scale than their STEM counterparts.

Gergen’s paper is wonderfully self-referential in this regard given its claim that social psychology is different from STEM and cannot issue the same kinds of law-like generalizations and predictions that one might find in physics and chemistry, not least because its very findings can alter the nature of the behavior it attempts to explain. What sociologists call reflexivity impacts the study of meaning in ways that are not so relevant to the study of molecules. And for this and other reasons the impact of work in social science is often more diffuse over time rather than being about breakthrough Eureka-like discoveries.

“Social Psychology as History” was published in one of the most prestigious and impactful social psychology journals. JPSP is published by the American Psychological Association and has published many of the most important papers in the field. Gergen’s article is controversial still and despite his warnings most of American social psychology has wedded itself to the claims that the STEM research model is applicable to their domain too. And this in turn has led to a replication crisis which maybe argues for Gergen’s original point.

So what does the fact that JPSP has a two-year impact factor of 7.673 actually tell you about the journal? A two-year moving window, designed with STEM research in mind, that certainly doesn’t include a nearly 50-year-old article with more than 3,000 citations. One might learn more about the journal from a comment made by Malcolm Gladwell who, on being asked where he would like to be buried, replied, “I’d like to be buried in the current-periodicals room, maybe next to the unbound volumes of the Journal of Personality and Social Psychology (my favorite journal).”

We at SAGE as publishers of social science have often debated this question and our role in perpetuating the problem. We have attempted to shift the emphasis by announcing five-year impact factors for the last couple of years and have created an annual prize for the most highly cited articles we have published over a ten-year period. While counting citations over a longer period of time will always have a place in measuring quality and impact, scholarly impact and significance cannot be restricted to this kind of measure alone.

The Metric Tide, an independent assessment of the role of metrics in assessing research led by James Wilsdon, among other things concluded that narrow metrics are not a responsible way to assess research without being supplemented by qualitative judgments such as peer review. The report advocated a move to responsible metrics – distinct from the broader moves towards responsible research more generally – which have the following characteristics:

• Robustness: basing metrics on the best possible data in terms of accuracy and scope;
• Humility: recognizing that quantitative evaluation should support – but not supplant – qualitative, expert assessment;
• Transparency: keeping data collection and analytical processes open and transparent, so that those being evaluated can test and verify the results;
• Diversity: accounting for variation by field, and using a range of indicators to reflect and support a plurality of research and researcher career paths across the system;
• Reflexivity: recognizing and anticipating the systemic and potential effects of indicators, and updating them in response.

The argument of course is not to abandon metrics entirely: we all need ways to filter knowledge claims to identify relevance and excellence. However, despite the diagnosis of the problem there is no consensus on the cure, as to how social science impact can best be demonstrated. In the absence of good alternatives, we default to the simple and dominant mode.

It is fitting that 1973 was not only the year in which Ken Gergen published his article in JPSP, but it also saw the launch of the Social Science Citation Index (SSCI). The Science Citation Index was created nearly a decade before. The SSCI will be 50 years old next year. Surely, we can do better than that in creating a high quality and impactful social science ecosystem.