Measuring all the Factors around Social Science Impact

SAGE Publishing workshop advances conversations on social science metrics

While the impact of research can be measured in many different ways, the traditional scientific form of measurement, Journal Citation Reports (traditionally known as Impact Factor), only measures citations by other scholars in scholarly publications. The impact that social science research makes beyond academia – on policy, the economy, and the public – while vast and varied (check out the Consortium of Social Science Association’s Why Social Science blog for some good examples), is not captured in any widely used way.

As part of a larger effort to support social scientists achieve and demonstrate impact, SAGE Publishing brought together 14 individuals who are both passionate about social science’s impact and intimately involved in improving its measurement for a one-day workshop held at Google’s main campus. With representation from Google Scholar, Clarivate, Altmetric, the Sloan Foundation, universities, and social science organizations, the group spent the day mapping out the impact landscape, making connections, and recommending opportunities for assessment of impact.  

The first step saw the group come to a consensus on who are the constituents of social science affected by existing and potential metrics. For example, while university administrators focus on more traditional journal metrics when it comes to tenure and promotion decisions, research funders are constantly looking for evidence that the projects they support are making a societal difference. Mission-based organizations like think tanks and NGOs are interested in research impact for policy change. Students are swayed to one discipline over another based on the impact they perceive – or don’t. And the general public feeds all of these decisions as it absorbs what is deemed valuable and influences decision makers.

The group then brainstormed data sources, key methods, and frameworks that are available, or could be made available, to help understand political and public impact. For example, government records could be mined to find evidence of research moving forward. Or we can reverse engineer research papers that have had substantial social influence to better understand the mechanisms underneath.

A number of groups currently are creating measuring tools. The Institute for Research on Innovation and Science, for example, is a group of universities developing tools that create reports for universities, data enclaves for researchers, and statistical evidence for policymakers. Open-source software from Project Jupyter, meanwhile, better measures impact by knocking down walls between programming languages and creating universal standards.  

The group made several recommendations for connecting their brainstormed data sources, methods, groups, and tools and for changing scholarly practices to better celebrate new forms of measurement and give credit at different points of the scholarly process, which will be published in the coming weeks here on Social Science Space.