Academic Funding

What Can REF Teach Us About Impact?

October 7, 2014 1185

Ever since reporting on the Research Excellence Framework began, a number of debates and controversies have followed. The REF has seemingly presented a number of problems for how academics see their role and how their work is perceived, and valued, by their managers. The new rules of this research assessment instrument appeared allusive for many academics, whilst a number of concerns were raised in relation to how ‘impact’ would be measured and how the REF might shape behaviors within the wider higher education landscape.

LSE Impact logo

This article by Tony Murphy and Daniel Sage originally appeared on the LSE Impact of Social Sciences blog as “Perceptions and ‘impacts’ of the REF: Key aim for next round should be to explore apprehension and minimise anxieties” and is reposted under the Creative Commons license (CC BY 3.0).

As curious academics, we wanted to understand what was being written about the REF and what this might signal in terms of the broader ‘impacts’ of the REF: from individual academics all the way up to the nature of academia itself. In order to do this, we undertook an analysis of the media: collecting media outputs on the REF over a two-year period to identify who was writing about the REF, what issues were being discussed alongside it, and the extent to which reporting was positive or negative.

Our findings raised some interesting ideas about the way in which the REF has been experienced by different types of authors. The results are published in a paper for the Journal of Higher Education Policy and Management. Our first key finding was – perhaps unsurprisingly – that reporting on the REF tended to be negative, regardless of who wrote the article. However, articles by senior academics (most often professors) were the most negative. We also found differences in the level of negativity according to subject area; for example, authors from the natural sciences were significantly more negative than social science authors.

murphyref2

We also found that those based at research intensive institutions were more likely to be writing about the REF in the first place; thus suggesting that they are more likely to have their voices heard. As we neared the REF submissions deadline we found a greater portion of articles citing the REF.  However, as we neared the REF submissions date we also found that reporting on the REF became less negative on an aggregate level – still negative, but not quite as negative as earlier in the cycle. Had academics become more at ease with the process or, perhaps, were they more focused on the task at hand?

Amongst the prominent themes discussed in the articles, ‘impact’ was the most frequent one. Others included ‘funding’ and ‘marketization.’ We also found that ‘institutional inequalities,’ ‘staff and management tensions,’ ‘intellectual freedom,’ ‘dishonest practice,’ ‘staff morale,’ ‘recruitment’ and ‘pressures to publish’ also figured strongly. For each of these themes we found, on an aggregate level, that the discussions around them were negative. The discussions around ‘staff morale’ and ‘staff and management tensions’ were the most negative on our scale. The level of negativity varied according to author background, as well as the issues being discussed.

murphyref1

From this, we argue that the REF has seemingly been experienced by different authors in a number of varying ways; different issues seem to be of concern for different types of authors, and some issues seem to attract a more negative discussion than others. We were particularly interested in the high level of negativity apportioned to professors, and the data seems to link this to a concern for ‘intellectual freedom.’ A further interesting finding is the comparatively heightened concern for ‘impact’ among junior academics and those at ‘teaching intensive’ institutions.

In all, we suggest that such findings offer a sense of some of the issues associated with the REF; they signal the range of concerns academics have around the REF and its processes. The perception that the REF can adversely shape behaviors is particularly noteworthy; we found discussions around ‘intellectual freedom,’ ‘dishonest practice’ and ‘disregard for certain research.’ We argue that this makes it ever-more important that we get it right when thinking about ‘impact’ in research and the pressures academics face, and how this might manifest itself in dubious or even dishonest research practices.

Accountability and transparency are, most academics would agree, vital within the management and funding of universities. Naturally, this extends to research activities. However, future research assessment exercises must be as transparent as possible, with clearly defined and accessible rules. We must also consider the possibility of adverse or even perverse outcomes when defining those rules. Exactly how the next research assessment process will pan-out is hard to predict with any certainty at this point in time.  However, as our research suggests, a key aim of the process should be to explore the broader apprehension that academics approach it with.  And, where possible, strive to minimize such anxieties.

***

For further details please see the original paper: Murphy, T., Sage, D. (forthcoming,2014). “Perceptions of the UK’s Research Excellence Framework 2014: a media analysis.” Journal of Higher Education Policy and Management. 36(6).


Tony Murphy is senior lecturer in criminology within the Department of Law and Criminology at Sheffield Hallam University. Daniel Sage is a doctoral researcher at the University of Stirling. In January 2015 he will take up a post in the School of Media, Culture and Society at the University of West of Scotland.

View all posts by Tony Murphy and Daniel Sage

Related Articles

Digital Scholarly Records are Facing New Risks
Research
May 21, 2024

Digital Scholarly Records are Facing New Risks

Read Now
New Fellowship for Community-Led Development Research of Latin America and the Caribbean Now Open
Academic Funding
May 14, 2024

New Fellowship for Community-Led Development Research of Latin America and the Caribbean Now Open

Read Now
Survey Suggests University Researchers Feel Powerless to Take Climate Change Action
Impact
April 18, 2024

Survey Suggests University Researchers Feel Powerless to Take Climate Change Action

Read Now
Three Decades of Rural Health Research and a Bumper Crop of Insights from South Africa
Impact
March 27, 2024

Three Decades of Rural Health Research and a Bumper Crop of Insights from South Africa

Read Now
Using Translational Research as a Model for Long-Term Impact

Using Translational Research as a Model for Long-Term Impact

Drawing on the findings of a workshop on making translational research design principles the norm for European research, Gabi Lombardo, Jonathan Deer, Anne-Charlotte Fauvel, Vicky Gardner and Lan Murdock discuss the characteristics of translational research, ways of supporting cross disciplinary collaboration, and the challenges and opportunities of adopting translational principles in the social sciences and humanities.

Read Now
Norman B. Anderson, 1955-2024: Pioneering Psychologist and First Director of OBSSR

Norman B. Anderson, 1955-2024: Pioneering Psychologist and First Director of OBSSR

Norman B. Anderson, a clinical psychologist whose work as both a researcher and an administrator saw him serve as the inaugural director of the U.S. National Institute of Health’s Office of Behavioral and Social Sciences Research and as chief executive officer of the American Psychological Association, died on March 1.

Read Now
New Feminist Newsletter The Evidence Makes Research on Gender Inequality Widely Accessible

New Feminist Newsletter The Evidence Makes Research on Gender Inequality Widely Accessible

Gloria Media, with support from Sage, has launched The Evidence, a feminist newsletter that covers what you need to know about gender […]

Read Now
0 0 votes
Article Rating
Subscribe
Notify of
guest

This site uses Akismet to reduce spam. Learn how your comment data is processed.

0 Comments
Inline Feedbacks
View all comments