Royal Society of Biology response to the Stern Review of the Research Excellence Framework

The review sought to “investigate ways in which a simpler, lighter-touch, system for the REF might be developed”. Led by Lord Stern, this review was the latest in a number of consultations on higher education and research policy.

The Royal Society of Biology has published their response to the Stern Review of the REF to which the BES contributed. Overall we remained supportive of the broad aims and objectives of the REF. However, the response identified a number of areas that need further guidance, clarification and in some instances amendment so as not to lead to unintended negative consequences.  Four of the stand out issues from our response were the clear support for retaining peer review of research quality (with some additional supporting measures); The burden placed on researchers, their teaching and career progression by the REF; the problem of credit for multi-author and interdisciplinary papers for early career researchers and collaborations; and finally, the negative effect of over focus on journal impact factor (IF).

To expand on a few of these highlighted issues staring with journal IF, it has been found that the problem of using journal IF as a measure of research importance is it is can lead to institutions distorting their investment priorities towards research fields that lend themselves to high impact factor publishing routes. What is needed is a system that incorporates a more sophisticated use of citation data and accounts for variables such as publication time. This should help ensure some important research areas that do not necessarily lend themselves to publication in high IF journals, such as taxonomy/systematics, don’t undergo a decline in funding in preference for broader popular topics.

While the REF has made improvements to ensure research benefits for society, policy, health wellbeing as well as the environment are better accounted for, there is still a need to understand and communicate the principles behind how to measure impact on these areas. As such there is a need to know how to recognise these impacts, how to capture them and how to widely communicate them.  This process of measuring research impact needs further guidance and to be clear, yet without becoming too tightly defined that it cannot be flexible enough to accommodate the different research outputs; be they more theoretical versus practical. It will also need to take account of whether research will have an immediate impact or a longer term impact, as well as the importance of research at the local versus international level.

These nuances all also serve to highlight the importance of the peer review process, and how difficult it would be to replace it with a metric assessment (quantitative indicator) only. A standalone metric assessment cannot provide a satisfactory assessment of the quality and impact of research. In addition, the over simplification might also see the loss of important information captured through the REF. This supports the view expressed in the independent review of metrics chaired by Prof. James Wilsdon that: “No set of numbers, however broad, is likely to be able to capture the multifaceted and nuanced judgements on the quality of research outputs that the REF process currently provides”.

There are also a number of other implications of REF, which is having a considerable influence on academic behaviour and not always with a positive outcome. For example, subjects which are naturally interdisciplinary can be penalised by the current REF if they fall between different units of assessment and the available funding is often low; for example any given ecological topic may be submitted to a number of different subject panels such as environmental science, biology or agriculture. Yet interdisciplinary research often plays an important role in policy discussions and thus has an important role to play outside of academia. There is also a danger that the REF could inhibit collaborations by over-incentivising REF performance of individual institutions and competition for papers authorship.

The REF is also perceived as having an impact on the role and focus of staff within research institutions with negative consequences such as there being extra pressure on research output, possibly the expense of quality teaching. It can also foster an over focus on REF performance when considering staff promotions, and encourage researchers to move too frequently to new institutions as well as unintended implications on the curriculum with a preference for staff working in fields more likely to produce REF-eligible outputs.

Finally, there are a few other key areas that the REF has a strong influence over such as publication strategies, potentially concentrating of research into an ever smaller number of institutions and even hindering development of both early stage careers.  Changes are needed to ensure these sometimes unintended consequences to do not continue to have problematic impacts so we will remain engaged with this issue as it develops.

Further details on the implications of the REF as well as for more information on the metrics and impacts can be seen in the response from the Royal Society of Biology to the Stern Review of the Research Excellence Framework. We will continue to engage with this review as it develops and, and welcome views from members – get in touch with your ideas or concerns.