Let a hundred flowers fade … The Stern Review
The publication of the report of the Stern Review, Research Excellence Framework (REF) Review: Building on Success and Learning from Experience , marks a new phase in the 30-year long audit of research in UK universities. It seems to recommend continuity – endorsement of dual-funding, maintenance of the REF2014 structure of assessment panels and peer review of outputs – but the small changes it does recommend will have profound implications.
The report describes UK research as highly productive, but acknowledges that the proportion of GDP devoted to research and development in the UK is among the lowest in the OECD. In the context of the vote to exit the EU and its consequences for research funding, we can be sure that the Government is unlikely to make up the shortfall but will want to get even more bang from its limited bucks. We can expect greater central direction of research.
The report provides the Government with the means to do so by strengthening the levers in the hands of University senior management, a group that has not, so far, indicated any strong commitments beyond the maximization of revenues and a willingness to fall in behind any government policy to maintain revenue (thus, Vice Chancellors have been silent on the freezing of the income threshold for the repayment of student loans, but are pleased to support a new Teaching Excellence Framework that will allow them to increase fees charged to students in line with inflation).
The report recommends a new institutional level for the assessment of research environment (probably at Main Panel level), with condensed statements by Units of Assessment (recommendations 8 and 9). This increases the significance of the strategic determination of research at university-level, which is further reinforced by the (partial) separation of impact case studies from specific Units of Assessment (recommendation 5).
Given that university strategies are already directed toward research-grant capture and Research Councils increasingly set (Government-approved) strategic priorities, there is convergence on ‘challenges’ that do not derive from the curiosity of researchers, but from the setting of targets and framing of topics by senior research policy-makers within universities and within Research Councils. Increasingly, all universities direct their research toward the same thematic priorities, including an emphasis on applied problem-solving interdisciplinary research. This undermines the significance of the very dual-funding system that the Review purports to uphold by aligning funding based on past research with funding for future research and puts both in the hands of managers.
The report suggests (recommendation 1) that all ‘research active’ staff should be submitted in the REF (which, of course, allows the creation of some staff as ‘teaching only’, a designation which is itself aligned with the fact that we shall now have ‘teaching only’ universities in the form of for-profit providers facilitated by the new Higher Education and Research Bill 2016-17, a development the report barely notices). Indeed, references to supporting excellence ‘wherever it is found’ when the major threat to many universities is not the administrative burdens of ‘interactions’ between the TEF and the REF to strengthen the relation between teaching and research (recommendation 12), but the fact that, for many universities, competition from new providers threatens to displace research (as is already occurring at London Met).
The report recommends (recommendation 2) that the number of staff submitted and their required number of outputs be disaggregated, but in a way that will not increase the burden on assessment panels. So, staff will submit between 0 and 6 outputs, but each Unit of Assessment will have an upper limit on outputs of ‘Staff FTEs submitted x 2’.
The report is against ‘gaming’ and ‘selectivity’ (evident in REF2014 by some institutions/UofAs to maximize the proportion of outputs at 3*/4*). But this proposal generalizes selectivity and punishes UofAs that previously practiced inclusion. An example will suffice to demonstrate what is at issue. Imagine two UofAs each with 20 members of staff. One submitted all 20 staff and, therefore, was evaluated on the basis of 80 outputs, the other submitted 10 and was evaluated on the basis of 40 outputs. Now, each must submit 20 members of staff, but submit only 40 outputs. Selectivity now operates by assigning fewer outputs to some staff in order to assign more outputs to other staff. The ‘virtuous’ UofA is now also constrained to a similar selectivity. ‘Gaming’ is only avoided by requiring everyone to play the same ‘bad’ game.
The negative consequences of selectivity for staff morale will also be accentuated because selectivity involves more intense performance management and internal processes of ‘picking winners’. Indeed, from an individual member of staff’s perspective there is little difference between not being included in the REF and being included but with no assigned outputs. The report suggests that some staff should be allowed to submit six publications, with the implication that it is because they have outstanding publications, but this requires other staff to have their submission reduced. ‘REF heroes’ require ‘REF zeros’. Now, selectivity is not based on the failure to meet the requirement to have four outputs, or to have a locally-assessed output score (the selectivity done at REF2014 to enhance the UofA profile), but will be done to allow another colleague to submit more outputs. This is a recipe for discord and division.
It is possible to put a positive ‘spin’ on the proposals, that it allows greater flexibility and allows colleagues to emphasize different activities. This has been argued on Twitter by Paul Kirby (@profpck). It seems to be a very unlikely outcome, not least because all the other recommendations diminish the autonomy of UofAs and reinforce managerial hierarchies. Indeed, some colleagues may already be living under a REF regime that is ‘scoring’ their outputs and checking those scores against data provided by a data analytics company, suggesting that their senior management had some inkling of the likely direction of the Stern Review (the clue lies in the scoring of ‘up to 6 items’).
Notwithstanding it is a consequence of their strategies, senior managers don’t like a transfer-market in REF ‘heroes’, so recommendation 3 is that outputs should not be portable. This seems to have been proposed without any wider consideration of the operation of the academic labour market. In effect, all new appointments to an institution come as ‘zeros’, needing to build a portfolio after appointment. This is especially problematic for early-career academics. They can seek to remain at their institution – perhaps one through which they have applied for a postdoc, on the basis of which they have a strong set of publications – but that is contingent on the institution being able to make the appointment permanent. Or, they must leave their outputs behind without having built up the academic capital typical of more senior colleagues and so are less able to build a portfolio during what remains of the REF-period. On the other hand, from the perspective of an externally appointing institution, it would be better to be appointed before publications come on stream. But on what basis would this be done other than ‘reputational’, including where someone has studied? So much for the concern with equalities.
Responding to the vote to exit the EU, the president of the Royal Society, Prof Sir Venki Ramakrishnan, recently argued for the need to recognize that UK research exists in a ‘global market for talent’. Perceptions that the UK was increasingly xenophobic, he argued, might put off overseas academics. However, they might also be put off by a public research system that is increasingly bureaucratized and performance managed. But what message do we give when we tell all new members of staff that they are ‘zeros’?