Why Social Science? Because It Makes an Outsized Impact on Policy

March 4, 2024 596

We know that social and behavioral science (SBS) has a hold on the conversation when institutional and government decision-makers parley over what goes into ‘policy.’ But oftentimes the SBS researchers whose own work goes into policy are unaware that they’re making an impact in the real world and are stymied from amplifying their findings or using them to advance their careers or fields.

With this in mind, Sage partnered with Overton to create Sage Policy Profiles, a free-to-use tool that enables researchers to discover the real-world impact – drawing from a pool of more than 10 million policy documents – of their work on policy, visualize, export, and share what they find. [Sage is the parent of Social Science Space, while access to Sage Policy Profiles is provided via Social Science Space.]

WhySocialScience logo_
This article was drawn from the Why Social Science? blog from the Consortium of Social Science Associations, and was originally titled “Because It Makes an Outsized Impact on Policy.”

In light of this launch, I sat down with Euan Adie, founder of Altmetric and Overton and currently Overton’s managing director, to learn more about the outsized impact that SBS makes on policy and his work creating tools to connect the scholarly and policy worlds.

What inspired you to found Overton? What was your ultimate goal?

I was always really interested in what goes into policy: How many decisions are backed by evidence? Who is influencing how that research is found and interpreted? We might imagine government experts, think tanks and corporate lobbyists are all interacting somehow – but how do you get involved as an academic? So one goal behind Overton is to make it a bit more transparent.

If you ask – as a researcher – how you can best engage with policymakers, you’ll get a very unsatisfactory answer about having to be very patient, but also ready to suddenly drop everything and put in lots of effort for potentially little return. All of which is correct but another goal at Overton is to make this process a little less painful for both sides. 

I love that it’s an area full of tricky challenges. COVID showed most people that it was useful to have relevant experts contributing to big policy decisions, and laws like the Evidence Act in the US show that lawmakers want that, too. But how do you translate the nuanced answers we’re trained to give as academics into something more pragmatic that can actually be used by a policymaker? That’s an exciting question.

Do you see disciplinary differences between how research impacts policy? What does that look like?

The biggest difference that jumps out at you is that the natural and physical sciences are rarely cited in policy while the social sciences are everywhere.

Can you share some data behind this?

Yes. To get some broad-brush numbers for this we took a random sample of ~ 1 million journal article records published between 2011 and 2020 from the OpenAlex database and then joined them with data from Overton to see which were cited by policy and how often. Fifty-eight percent of the records in our set had scholarly citations and 4 percent have policy citations.

Those policy citations weren’t evenly distributed across subjects. We used OpenAlex’s “concepts” taxonomy to group articles into different subject areas.  

In all of the areas when you compare mean scholarly citations to mean policy citations the scholarly citations mean is higher, which makes sense: articles get cited by other scholars far more than they do in policy. 

But the ratio between the two changes from one subject to another. Of the areas we looked at, Chemistry, Materials Science, and Physics publications have the highest average number of scholarly citations (relative to other subjects) but the lowest average number of policy citations. 

In Business & Economics things are flipped: the subject has few scholarly citations (generally, and relative to other subjects) while receiving many more policy citations. The same holds true for Political Science, Psychology & Sociology.

We found something similar when looking at the Scopus journal categories of all articles cited in policy as part of a broader analysis that was published in QSS

Why do you think this is?

There may simply be more policy about education, social care, prison welfare or macroeconomics than there is about organic chemistry or quantum physics.

Anecdotally for lots of different reasons some policy areas are better at finding, interpreting, using and citing evidence than others – public health or food safety, for example, often have associated technical agencies and policymakers who are themselves experts in the field.

Other areas are broader and you might have policymakers who are generalists trying to collect information as best they can on a tight schedule.

And that said…. there are biases and caveats in the data too. We’re only looking at papers explicitly cited in policy documents, so have to acknowledge that the numbers will be biased towards subject areas where it’s made more clear where the evidence is coming from, where there’s more transparency about what evidence was used in each document. 

According to your data, what policy sources across the globe most use SBS research? What about the most US-based sources?

Intergovernmental organizations (IGOs) are big users of evidence, and very important as knowledge brokers sitting in-between the academic and policy worlds. The OECD, World Bank, UNESCO and WHO are all big users of social sciences research.

Federal government aside, two of the biggest users in the US are actually think tanks: RAND and Brookings. Social sciences research also gets picked up as evidence by the Congressional Research Service or used by House and Senate committees.

Can you break down your data by topic area? What topics are most cited according to your data?

We can! In terms of which topics are most cited by policy documents, when we look at Scopus journal subjects, the top ones are:

  • Medicine — Public Health, Environmental and Occupational Health
  • Economics —  Economics and Econometrics
  • Life Sciences —  Ecology, Evolution, Behavior and Systematics
  • Social Sciences —  Sociology and Political Science
  • Medicine —  Psychiatry and Mental Health

To some degree that fits with the topics of the policy documents themselves. We can get a feel for that by looking at what proportion of policy documents fit into the different UN Sustainable Development Goals (SDGs) with the caveat that they are development-focused. The three most common SDGs are:

  • Good Health and Well-being
  • Decent Work and Economics Growth
  • Quality Education

How would you say Sage Policy Profiles differs from other products you have put out?

 We spent much more time on this app thinking about the user experience for individual researchers, partly after seeing how much Sage cares about the same thing, which I think has really paid off.

Being focused on one person’s data at a time was also new and opened up some opportunities that we couldn’t do otherwise, such as letting the user curate their citations and mentions.

Now looking beyond these tools, what do you think is needed to strengthen the ties between researchers, academic institutions, and policymakers?

 There’s really interesting work happening on the demand side of evidence-based policy that I think has the potential to do a lot of good.

Simple sounding things like making sure academics and funders know what topics their government is actually looking for help on: Learning Agendas in the US and the Areas of Research Interest in the UK are two good examples of this, and all the work Transforming Evidence has done on the latter is a highlight.

Policymakers and academics work on different timescales and have different incentives, and that’s something else that I think needs addressing. Arguably the current system we have for assessing researchers is pretty broken, but we don’t need to overhaul the whole thing to make meaningful improvements to it for people who want their work to be useful: there are incremental improvements we might make.

Camille Gamboa is associate vice president, corporate communications at Sage. She sits on the Banned Books Task Force for the Association of American Publishers, convenes representatives from across the trans-Atlantic to support a healthy future for social and behavioral science, and is a regular blogger for the Charleston Hub. Follow @CamilleGamboa on X

View all posts by Camille Gamboa

Related Articles

To Better Forecast AI, We Need to Learn Where Its Money Is Pointing
April 10, 2024

To Better Forecast AI, We Need to Learn Where Its Money Is Pointing

Read Now
Free Online Course Reveals The Art of ChatGPT Interactions
March 28, 2024

Free Online Course Reveals The Art of ChatGPT Interactions

Read Now
The Importance of Using Proper Research Citations to Encourage Trustworthy News Reporting
February 26, 2024

The Importance of Using Proper Research Citations to Encourage Trustworthy News Reporting

Read Now
Why Don’t Algorithms Agree With Each Other?
February 21, 2024

Why Don’t Algorithms Agree With Each Other?

Read Now
A Behavioral Scientist’s Take on the Dangers of Self-Censorship in Science

A Behavioral Scientist’s Take on the Dangers of Self-Censorship in Science

The word censorship might bring to mind authoritarian regimes, book-banning, and restrictions on a free press, but Cory Clark, a behavioral scientist at […]

Read Now
SSRC Links with U.S. Treasury on Evaluation Projects

SSRC Links with U.S. Treasury on Evaluation Projects

Thanks to a partnership between the SSRC and the US Department of the Treasury, two new research opportunities in program evaluation – the Homeowner Assistance Fund Project and the State and Local Fiscal Recovery Funds Project – have opened.

Read Now
NSF Responsible Tech Initiative Looking at AI, Biotech and Climate

NSF Responsible Tech Initiative Looking at AI, Biotech and Climate

The U.S. National Science Foundation’s new Responsible Design, Development, and Deployment of Technologies (ReDDDoT) program supports research, implementation, and educational projects for multidisciplinary, multi-sector teams

Read Now
0 0 votes
Article Rating
Notify of

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Inline Feedbacks
View all comments