The wait is over, the results of the Research Excellence Framework (REF) will be released to the world Thursday. In the run up to the big announcement it isn’t surprising that there is plenty of commentary about the REF, its perceived strengths and weaknesses, its costs and benefits, the funding outcomes, and of course the future of and planning for the next exercise. An example of the latter is a letter from the presidents of the British Academy and Royal Society in last week’s THE calling for a thorough evaluation of the costs of the REF, to inform future planning. In my team, we have actually been thinking about the future of the exercise for close to two years already. We believe an essential starting point is gathering robust evidence about the process that we are just completing, as well as getting up-to-date evidence on wider questions of research assessment. So, working with the other UK Funding Bodies, we have an extensive and detailed programme of work underway, or due to be commissioned, that will provide a solid foundation on which to debate the future during 2015.
While all of these activities are in the public domain, the aim of this post is to present them together, to give a sense of the overall programme.
- One focus of the evaluation is on the impact element of the REF. This element is new, not just for the UK but globally, so it is essential that we subject what we have done to suitable scrutiny. During 2013 and 2014 we commissioned RAND Europe to investigate the costs, benefits and implications for institutions preparing submissions to the exercise. And now, RAND are carrying out a second phase of the evaluation working with REF panel members to evaluate the assessment process for impact. These studies will provide an in depth review of the impact element from all angles.
- At the same time, we have also commissioned Digital Science and Kings College London to carry out a synthetic analysis of the case studies submitted to the REF, that will provide not only evidence on the impact that research in HEIs makes, but will also provide valuable information on the types of impact that feature in the REF, and how impact varies by discipline.
- To further our understanding of the implications of the REF within institutions, at the beginning of this year we also invited feedback from HEIs on the process. More than 100 HEIs (out of the 154 who submitted) responded, and provided a wealth of information on costs, benefits, problems and solutions.
- We are also seeking feedback from REF panel members, and are running a series of focus groups with them. These are already providing valuable insight.
- While the impact studies described already are examining the costs associated with that element, we have also commissioned from Technopolis a further study to estimate the costs and benefits of the rest of the exercise. A particular focus will be the costs of the provisions made for special staff circumstances. These provisions have been welcomed by institutions and researchers alike, but we know that there have been associated cost implications.
- We are also about to commission work to examine the position of multi- and inter-disciplinary research in the REF, as a part of wider work, some of it planned jointly with RCUK. We are also doing some work to look at the extent to which outputs submitted to the REF reflect collaborations between institutions.
Taken together these evaluations will provide a rounded and complete evaluation of the REF. But it is also important that we consider wider issues beyond the current exercise. A central part of this is the independent review on the use of metrics in research assessment that is being led by professor James Wilsdon. While this study is broader than just a consideration of the use of metrics in national research assessments, it is going to provide important evidence to inform the thinking about future REF exercise.
Once the solid foundation of evidence is built during the course of 2015, we can then begin the debate about the future, with researchers and HEIs, of course, at its heart.