A version of this article first appeared in Funding Insight on 9th March 2018 and is reproduced with kind permission of Research Professional. For more articles like this, visit www.researchprofessional.com
“Just when I thought I was out, they pull me back in!”
My previous post posed a question about whether applying for research funding was worth it or not, and concluded with a list of questions to consider to work out the answer. This follow-up is a list of costs and benefits associated with applying for external research funding, whether successful or unsuccessful. Weirdly, my list appears to contain more costs than benefits for success and more benefits than costs for failure, but perhaps that’s just me being contrary…
If you’re successful:
Benefits….
You get to do the research you really want to do
In career terms, whether for moving institution or internal promotion, there’s a big tick in the box marked ‘external research funding’.
Your status in your institution and within your discipline is likely to rise. Bringing in funding via a competitive external process gives you greater external validation, and that changes perceptions – perhaps it marks you out as a leader in your field, perhaps it marks a shift from career young researcher to fulfilling your evident promise.
Success tends to begat success in terms of research funding. Deliver this project and any future application will look more credible for it.
Costs…
You’ve got to deliver on what you promised. That means all the areas of fudge or doubt or uncertainty about who-does-what need to be sorted out in practice. If you’ve under-costed any element of the project – your time, consumables, travel and subsistence – you’ll have to deal with it, and it might not be much fun.
Congratulations, you’ve just signed yourself up for a shedload of admin. Even with the best and most supportive post-award team, you’ll have project management to do. Financial monitoring; recruitment, selection, and line management of one or more research associates. And it doesn’t finish when the research finishes – thanks to the impact agenda, you’ll probably be reporting on your project via Researchfish for years to come.
Every time any comparable call comes round in the future, your colleagues will ask you give a presentation about your application/sit on the internal sifting panel/undertake peer review. Once a funding agency has given you money, you can bet they’ll be asking you to peer review other applications. Listed as a cost for workload purposes, but there are also a lot of benefits to getting involved in peer reviewing applications because it’ll improve your own too. Also, the chances are that you benefited from such support/advice from senior colleagues, so pay it forward. But be ready to pay.
You’ve just raised the bar for yourself. Don’t be surprised if certain people in research management start talking about your next project before this one is done as if it’s a given or an inevitability.
Unless you’re careful, you may not see as much recognition in your workload as you might have expected. Of course, your institution is obliged to make the time promised in the grant application available to you, but unless you’ve secured agreement in advance, you may find that much of this is taken out of your existing research allocation rather than out of teaching and admin. Especially as these days we no longer thing of teaching as a chore to buy ourselves out from. Think very carefully about what elements of your workload you would like to lose if your application is successful.
The potential envy and enmity of colleagues who are picking up bits of what was your work.
If you’re unsuccessful…
Benefits…
The chances are that there’s plenty to be salvaged even from an unsuccessful application. Once you’ve gone through the appropriate stages of grief, there’s a good chance that there’s at least one paper (even if ‘only’ a literature review) in the work that you’ve done. If you and your academic colleagues and your stakeholders are still keen, the chances are that there’s something you can do together, even if it’s not what you ideally wanted to do.
Writing an application will force you to develop your research ideas. This is particularly the case for career young researchers, where the pursuit of one of those long-short Fellowships can be worth it if only to get proper support in developing your research agenda.
If you’ve submitted a credible, competitive application, you’ve at least shown willing in terms of grant-getting. No-one can say that you haven’t tried. Depending on the pressures/expectations you’re under, having had a credible attempt at it buys you some license to concentrate on your papers for a bit.
If it’s your first application, you’ll have learnt a lot from the process, and you’ll be better prepared next time. Depending on your field, you could even add a credible unsuccessful application to a CV, or a job application question about grant-getting experience.
If your institution has an internal peer review panel or other selection process, you’ve put you and your research onto the radar of some senior people. You’ll be more visible, and this may well lead to further conversations with colleagues, especially outside your school. In the past I’ve recommended that people put forward internal expressions of interest even if they’re not sure they’re ready for precisely this reason.
Costs…
You’ve just wasted your time – and quite a lot of time at that. And not just work time… often evenings and weekends too.
It’ll come as a disappointment, which may take some time to get over
Even if you’ve kept it quiet, people in your institution will know that you’ve been unsuccessful.
I’ve written two longer pieces on what to do if your research grant application is unsuccessful, which can be found here and here.
A version of this article first appeared in Funding Insight on 6th March 2018 and is reproduced with kind permission of Research Professional. For more articles like this, visit www.researchprofessional.com
Success rates are low and applications are more and more time consuming to write. Is it worth it? Here’s a quick list of considerations that might help you reach a better decision.
While the latest success rates from UK research councils showed a very modest overall improvement after five consecutive annual falls, most observers regard this as a blip rather than as a sign of better times to come. Outside the Research Councils, success rates are often even lower, with some social science/humanities fellowship schemes having single digit success rates.
While success rates have fallen, demands on applicants have steadily risen. The impact agenda has brought first the impact summary and then the pathways to impact statement, and more recently we’ve seen greater emphasis on data management plans and on detailed letters of support from project partners that require significant coordination to obtain. It would be one thing if it were just a question of volume – if you want a six or seven figure sum of what’s ultimately public money, it’s not unreasonable to be asked to work for it. But it’s not just that, it’s also the fiddly nature of using JeS and understanding funder requirements. I’m forever having to explain the difference between the pathways to impact and the impact summary, and there are lots of little quirks and hidden sections that can trip people up.
But beyond even that, there’s the institutional effort of internal peer review from research development staff and senior and very busy academic staff. Whether that’s an internal review mandated by the research council – shifting the burden of review onto institutions – or introduced as a means of improving quality, it’s another cost.
Given the low success rates, the effort and time required, and the opportunity costs of doing so, are we wasting our time? And how would we know?
The research
Do you need funding to do the research? If not, might it be a better idea just to get on with it, rather than spend a month writing an application and six months waiting for a response? And if you only need a small amount of funding, consider a smaller scheme with a less onerous application process.
Do you have a clear idea of what you want to achieve? If you can’t identify some clear research questions, and what your project will deliver, the chances are it needs more thinking through before it’s ready to be turned into an application.
Are you and your team passionate and enthused and excited about your proposal? If you’re not, why should anyone else be?
Is your research idea competitive? That’s not the same question as ‘is it good’? To quote a research director from a Canadian Research Council – it’s not a test, it’s a contest. Lots and lots and lots of good ideas go unfunded. Just because you could get something in that’s in scope and has at least some text in every box doesn’t mean you should.
Is your research idea significant? In other words, does it pass the ‘so what, who cares’ test? My experience on an NIHR funding panel is that once the flawed are eliminated, funding is a battle of significance. Is your research idea significant, would others outside your field regard it as significant, and can you communicate its significance?
Your motivations
Are they intrinsic to the research – to do with the research and what you and your team want to discover and achieve and contribute…. or are they extrinsic?
Are you applying for funding because you want promotion? When you come and talk to me and my colleagues about ‘applying for funding’ but have less a coherent project and more of a list of random keywords, don’t think we don’t know.
Is it because you/your research group/school is being pressured to bring in more funding? Football manager Harry Redknapp’s tactical instructions to a substitute apparently once consisted of “just flipping run around a bit” (I paraphrase) and I sometimes worry that in some parts of some institutions that’s what passes for a grant capture strategy that values activity over outcomes.
Is it because you want to keep researchers on fixed term contracts/your promising PhD student in work? That’s a laudable aim, but without the right application and idea, you risk giving them false hope if the application is just to do more of the same with the same people.
Practical considerations
Do you have the time you need to write a competitive application? Just as importantly, do your team? Will they be able to deliver on the bits of the application they’ll need to write? As Yoda said, “do or do not, there is no try” (Lucas, 1980). If you can’t turn your idea into a really well written, competitive, proposal in time, perhaps don’t.
Do you have your ducks in a row? Your collaborators and co-Is, your industry, government, or third sector partners lined up and on board? Are your impact plans ready? Or are you still scratching around for project partners while your competitors are polishing the fourth iteration of the complete application? Who are your rivals for this funding? Not relevant for ‘open’ calls, but for targeting schemes, who else is likely to be going for this?
Does what you want to do fit the call you’re considering applying for? Read the call, read it again, and then speak to your friendly neighbourhood Research Development professional and see if your understanding of the call matches hers. Why? Because it’s hard for researchers to read a call for proposals without seeing it through the lens of their own research priorities. Make sure others think it’s a good fit – don’t trust yourself or your co-Is to make that decision alone.
Is this the best use of your time right now? Might your time be better spent on impact, publishing papers from the last project, revising a dated module, running professional development courses?
A companion piece on the costs and benefits to researchers of applying for funding will be republished here next week.
My headline, unfortunately, is a classic QTWTAIN (‘question to which the answer is no’) because I can’t see any evidence that the employers want to negotiate or seek alternatives or engage in any meaningful way. You can find an admirably clear (and referenced) summary of the current situation at the time of writing here.
And if you want to read my previous wibblings from a previous dispute about why you should join the union, see the second half of this post.
All I’ll add is that we’ve been here before as regards pensions cuts… again and again and again… only previously it’s been salami slices, or at least compared to what’s being proposed now. These previous changes, we were told, would put the scheme back on the right track, and were necessary due to increased life expectancy etc and so on. So my question is… were those previous claims about past changes just straightforward lies, or have things got worse? And if they’ve got worse, is that wider economic conditions, or incompetence? And either way, why are the people responsible taking huge pay increases? Why is my pension scheme on the way to becoming a regular in the pages of Private Eye?
Anyway… I wanted to talk about deliberative democracy. I listened to a really interesting Reasons to be Cheerfulpodcast (presented by Ed Miliband and Geoff Lloyd) on deliberative democracy the other week. If we ask everyone what they think on a particular topic, the problem is that not everyone will be equally well informed, will have the necessary time to follow the arguments and find the evidence, or will come to the topic with an open mind. The idea of deliberative democracy is finding a small, representative group, giving them full access to the evidence and the arguments and expertise, and then, through deliberation, work towards a consensus decision if possible.
Trial by jury follows this model very closely, though we don’t typically think of a jury as an expression of democracy. These are twelve ordinary people, selected at random (with some exceptions and criteria), and trusted to follow the arguments in a criminal trial. But we regard this as fair, and as legitimate, and my perception is that there’s widespread faith in trial by jury as an institution.
Could we extend this to other issues? For example, the current strike action about cuts to the USS pension scheme. At the moment I’m reading a lot of criticism about the methods of calculation, the underpinning assumptions, and some very questionable motivations and methods of reaching and spinning decisions by Universities UK. But some of that criticism comes from people who aren’t experts in this area, but have relevant expertise in other related areas, or in areas that share a skill set. Such cognate-experts might well be right, but equally there might be good explanations for some of the peculiar-looking assumptions. In keeping with the Dunning-Kruger Effect, might such people be overestimating their own expertise and underestimating those of genuine experts? I don’t know.
Hence my interest in deliberative democracy… get a representative group of pension scheme members (academic and APM, a range of ages (including PhD students and retired staff), union and non-union members, a range of seniority and experience and subject area/specialism), give them access to experts and evidence, and let’s see what they come back with. A report from such a group that contends that, yes, the pension scheme is in trouble and an end to defined benefit is the only thing that will keep it sustainable, would have credibility and legitimacy. On the other hand, a report that came back with other options and which denied that case for the necessity for such a drastic step, would also be persuasive. This would be a decision by my peers who have taken more time and more trouble than I have, who have access to expertise and arguments and evidence, and who I would therefore trust.
“Down with this sort of thing!”
I strongly suspect that we have two very polarised actuarial valuations of the scheme – one, from the employers, which seems to me to be laughably flawed (but again, Dunning Kruger… what do I know?), and another, from UCU, which may turn out to be laughably optimistic. Point is, I don’t know, and I don’t want to make the mistake of assuming that the truth must lie somewhere in between.
One objection is that this might be little different to recent accusations about university Vice Chancellors sitting on the committees that set their salaries. However, a range of ages and career stages could mitigate against this – younger group members would surely resist any attempts to allow the scheme to limp on until older members are likely to be retired but which would leave them with little or nothing. We can also include information about affordability and HE finance in general to ensure that we don’t end up with recommendations that are completely unaffordable.I’d also like to think that those who chose careers in academia or in university management – in most cases ahead of more lucrative careers – have a commitment to the sector and its future.
And no-one’s saying that the report of such a group need be binding, but, a properly constituted group undertaking deliberative work with access to evidence and expertise would carry a great deal of authority and would be hard to be simply set aside. It’s an example of what John Rawls called ‘pure procedural justice‘. Its outcome is fair because it is set up and operates in a way that’s fair.
So I guess that’s my challenge to Universities UK and (to a lesser extent) the UCU too. If, UUK, your argument is that ‘There Is No Alternative’ (TINA) – which we’ve heard before, ad nausem – let’s see if that’s really the case. Their complete refusal to engage on the issue of the ending of defined benefit doesn’t bode well here, nor does the obvious disingenuous of offering “talks” while refusing to negotiate on the issue the strike is actually about. But let’s see if UCU’s claims bear scrutiny too. No-one is immune from wishful thinking, and some elements within UCU seem to enjoy being on strike a bit too much for my liking.
Because, frankly, I’d quite like (a) to get back to work; and (b) have some sort of security in retirement, and the same for generations of academics and APM staff to come.
This article first appeared in Funding Insight on 11th May 2017 and is reproduced with kind permission of Research Professional. For more articles like this, visit www.researchprofessional.com
* * * * * * * * * * * * * * * * * * ** *
Should I resubmit my unsuccessful research grant application?
No.
‘No’ is the short answer – unless you’ve received an invitation or steer from the funder to do so. Many funders don’t permit uninvited resubmissions, so the first step should always be to check your funder’s rules and definitions of resubmission with your research development team.
To be, or not to be
That’s not to say that you should abandon your research proposal – more that it’s a mistake to think of your next application on the same or similar topic as a resubmission. It’s much better – if you do wish to pursue it – to treat it as a fresh application and to give yourself and your team the opportunity to develop your ideas. It’s unlikely that nothing has changed between the date of submission and now. It’s also unlikely that nothing could be improved about the underpinning research idea or the way it was expressed in the application.
However, sometimes the best approach is to let an idea go, cut your losses, avoid the sunk costs fallacy. Onwards and upwards to the next idea. I was recently introduced to the concept of a “negative CV”, which is the opposite of a normal CV, listing only failed grant applications, rejected papers, unsuccessful conference pitches and job market rejections. Even the most eminent scholars have lengthy negative CVs, and there’s no shame in being unsuccessful, especially as success rates are so low. It’s really difficult – you’ve got your team together, you’ve been through the discussions and debates and the honing of your idea and then the grant writing, and then the disappointment of not getting funded. It’s very definitely worth having meetings and discussion to see what can be salvaged and repurposed – publishing literature reviews, continuing to engage with stakeholders etc. It’s only natural to look for some other avenue for your work, but sometimes it’s best to move on to something else.
Here are two bits of wisdom that are both true in their own way:
If at first you don’t succeed, try, try try again (William Edward Hickson)
The definition of insanity is doing the same thing over and over but expecting different results (disputed- perhaps Einstein or Franklin, but I reckon US Narcotics Anonymous)
So what should you do? What factors should you consider in deciding whether to rise from the canvas like Rocky, or instead emulate Elsa and Let It Go?
What being unsuccessful means… and what it doesn’t
As a Canadian research council director once said, research funding is a contest, not a test. Research funding is a limited commodity, like Olympic medals, jobs, and winning lottery tickets. It’s not an unlimited commodity like driving licenses or PhDs, commodities which everyone who reaches the required standard can obtain. Sometimes I think researchers confuse the two – if the driving test examiner says I failed on my three point turn, if I get it right next time (and make no further mistakes) I’ll pass. But even if I respond adequately to all of the points made in the referees’ comments, there’s still no guarantee I’ll get funded. The quality of my driving in the morning doesn’t affect your chances of passing your test in the afternoon, but if too many applications are better than yours, you won’t get funded. And just as many recruitment exercises produce more appointable candidates than posts, so funding calls attract far more fundable applications than the funds available.
Sometimes referees’ comments can be misinterpreted. Feedback might list the real or perceived faults with the application, but (once the fundamentally flawed have been excluded) ultimately it’s a competition about significance. What significance means is defined by the funder and the scheme and doesn’t necessarily mean impact – it could be about academic significance, contribution to the field and so on.
As a public panel member for an NIHR scheme I’ve seen this from the inside – project proposals which are technically competent, sensible and feasible. Yet either because they fail to articulate the significance or because their research challenge is just not that significant an issue, they don’t get funded because they’re not competitive against similarly competent applications taking on much more significant and important research challenges. Feedback is given which would have improved the application, but simply addressing that feedback will seldom make it any more competitive.
When major Research Centre calls come out, I often have conversations with colleagues who have great ideas for perfectly formed projects which unfortunately I don’t think are significant enough to be one of three or four funded across the whole of social sciences. Ideally the significance question, the “so what/who cares?” question should be posed before applying in the first place, but you should definitely look again at what was funded and ask it again of your project before considering trying to rework it.
Themed Calls Cast a Long Shadow
One of the most dispiriting grant rejection experiences is rejection from a targeted call which seemed perfect. It’s not like an open call where you have to compete with rival bids on significance from all across your research council’s remit – rather, the significance is already recognised.
Yet the reality is that narrower calls often have similarly low success rates. Although they’re narrower, everyone who can pile in, does pile in. And deciding what to do next is much harder. Themed calls cast a long shadow – if as a funder I’ve just made a major investment in field X through niche call Y, I’m not sure how I’m going to feel about an X-related application coming back in through the open call route. Didn’t we just fund a lot of this stuff? Should we fund more, especially if an idea like this was unsuccessful last time? Shouldn’t we support something else? And I think this effect might be true even with different funders who will be aware of what’s going on elsewhere. If a tranche of projects in your research area have been funded through a particular call, it’s going to be very difficult to get investment through any other scheme anytime soon.
Switching calls, Switching funders
An exception to this might be the Global Challenges Research Fund or perhaps other areas where there’s a lot of funding available (relatively speaking) and a number of different calls with slightly different priorities. Being unsuccessful with an application to an open call or a broader call and then looking to repurpose the research idea in response to a narrower themed call is more likely to pay off than the other way round, moving from a specific call to a general one. But even so, my advice would be to ban the “r” word entirely. It’s not a ‘resubmission’, it’s an entirely new application written for a different funding scheme with different priorities, even if some of the underlying ideas are similar.
This goes double when it comes to switching funders. A good way of wasting everyone’s time is trying to crowbar a previously unsuccessful application into the format required by a different funder. Different funders have different priorities and different application procedures, formats and rules, and so you must treat it as a fresh application. Not doing so is a bit like getting out some love letters you sent to a former paramour, changing the name at the top, and reposting them to the current object of your affections. Neither will end well.
The Leverhulme Trust are admirably clear on this point, they’re “keen to avoid assuming the role of ‘funder of last resort’; that is, of routinely providing support for proposals which have been fully matched to the requirement of another funding agency, but have failed to win support on the grounds of either lack of quality or insufficient available funds.” If you’re going to apply to the Leverhulme Trust, for example, make it a Leverhulme-y application, and that means shifting not just the presentational style but also the substance of what you’re proposing.
Whatever the change, forget any notion of resubmission if you’re taking an idea from one call to another. Yes, you may be able to reuse some of your previous materials, but if you submit something clearly written for another call with the crowbar marks still visible, you won’t get funded.
The Five Stages of Grant Application Failure
I’m reluctant to draw this comparison, but I wonder if responding to grant application rejection is a bit like the Kubler-Ross model of grief (denial, anger, bargaining, depression, and acceptance). Perhaps one question to ask yourself is if your resubmission plans are coming from a position of acceptance – in which case fine, but don’t regard it as a resubmission – or a part of the bargaining stage. In which case…. perhaps take a little longer to decide what to do.
In my previous post I wrote about the Stern Review, and in particular the portability issue – whereby publications remained with the institution where they were written, rather than moving institutions with the researcher – which seemed by some distance the most vexatious and controversial issue, at least judging by my Twitter feed.
Since then there has been a further announcement about a forthcoming consultation exercise which would seek to look at the detail of the implementation of the Stern Review, giving a pretty clear signal that the overall principles and rationale had been accepted, and that Lord Stern’s comments that his recommendations were meant to be taken as a whole and were not amenable to cherry picking, had been heard and taken to heart.
Today – only ten days or so behind schedule – the consultation has been launched. It invites “responses from higher education institutions and other groups and organisations with an interest in the conduct, quality, funding or use of research”. In paragraph 15, this invitation is opened out to include “individuals”. So as well as contributing to your university response, you’ve also got the opportunity to respond personally. Rather than just complain about it on Twitter.
Responses are only accepted via an online form, although the questions on that online form are available for download in a word document. There are 44 questions for which responses are invited, and although these are free text fields, the format of the consultation is to solicit responses to very specific questions, as perhaps would be expected given that the consultation is about detail and implementation. Paragraph 10 states that
“we have taken the [research excellence] framework as implemented in 2014 as our starting position for this consultation, with proposals made only in those areas where our evidence suggests a need or desire for change, or where Lord Stern’s Independent Review recommends change. In developing our proposals, we have been mindful of the level of burden indicated, and have identified where certain options may offer a more deregulated approach than in the previous framework. We do not intend to introduce new aspects to the assessment framework that will increase burden.”
In other words, I think we can assume that 2014 plus Stern = the default and starting position, and I would be surprised if any radical departures from this resulted from the consultation. Anyone wanting to propose something radically different is wasting their time, even if the first question invites “comments on the proposal to maintain an overall continuity of approach with REF 2014.”
So what can we learn from the questions? I think the first thing that strikes me it’s that it’s a very detailed and very long list of questions on a lot of issues, some of which aren’t particularly contentious. But it’s indicative of an admirable thoroughness and rigour. The second this is that they’re all about implementation. The third is that reduction of burden on institutions is a key criterion, which has to be welcome.
Units of Assessment
It looks as if there’s a strong preference to keep UoAs pretty much as they are, though the consultation flags up inconsistencies of approach from institutions around the choice of which of the four Engineering Panels to submit to. Interestingly, one of the issues is comparability of outcome (i.e. league tables) which isn’t technically supposed to be something that the REF is concerned with – others draw up league tables using their own methodologies, there’s no ‘official’ table.
It also flags up concerns expressed by the panel about Geography and Archaeology, and worries about forensic science, criminology and film and media studies, I think around subject visibility under current structures. But while some tweaks may be allowed, there will be no change to the current structure of Main Panel/Sub Panel, so no sub-sub-panels, though one of the consultation possibilities is is about sub-panels setting different sub-profiles for different areas that they cover.
Returning all research active staff
This section takes as a starting point that all research active staff will be returned, and seeks views on how to mitigate game-playing and unintended consequences. The consultation makes a technical suggestion around using HESA cost centres to link research active staff to units of assessment, rather than leaving institutions to the flexibility to decide to – to choose a completely hypothetical example drawn in no way from experience with a previous employer – to submit Economists and Educationalists into a beefed up Business and Management UoA. This would reduce that element of game playing, but would also negatively effect those whose research identity doesn’t match their teaching/School/Department identity – say – bioethicists based in medical or veterinary schools, and those involved in area studies and another discipline (business, history, law) who legitimately straddle more than one school. A ‘get returned where you sit’ approach might penalise them and might affect an institution’s ability to tell the strongest possible story about each UoA.
As you’d expect, there’s also an awareness of very real worries about this requirement to return all research active staff leading to the contractual status of some staff being changed to teaching-only. Just as last time some UoAs played the ‘GPA game’ and submitted only their best and brightest, this time they might continue that strategy by formally taking many people out of ‘research’ entirely. They’d like respondents to say how this might be prevented, and make the point that HESA data could be used to track such wholesale changes, but presumably there would need to be consequences in some form, or at least a disincentive for doing so. But any such move would intrude onto institutional autonomy, which would be difficult. I suppose the REF could backdate the audit point for this REF, but it wouldn’t prevent such sweeping changes for next time. Another alternative would be to use the Environment section of the REF to penalise those with a research culture based around a small proportion of staff.
Personally, I’m just unclear how much of a problem this will be. Will there be institutions/UoAs where this happens and where whole swathes of active researchers producing respectable research (say, 2-3 star) are moved to teaching contracts? Or is the effect likely to be smaller, with perhaps smaller groups of individuals who aren’t research active or who perhaps haven’t been producing being moved to teaching and admin only? And again, I don’t want to presume that will always be a negative move for everyone, especially now we have the TEF on the horizon and we are now holding teaching in appropriate esteem. But it’s hard to avoid the conclusion that things might end up looking a bit bleak for people who are meant to be research active, want to continue to be research active, but who are deemed by bosses not to be producing.
Decoupling staff from outputs
In the past, researchers were returned with four publications minus any reductions for personal circumstances. Stern proposed that the number of publications to be returned should be double the number of research active staff, with each person being about to return between 0 and 6 publications. A key advantage of this is that it will dispense with the need to consider personal circumstances and reductions in the number of publications – straightforward in cases of early career researchers and maternity leaves, but less so for researchers needing to make the case on the basis of health problems or other potentially traumatic life events. Less admin, less intrusion, less distress.
One worry expressed in the document is about whether this will allow panel members to differentiate between very high quality submissions with only double the number of publications to be returned. But they argue that sampling would be required if a greater multiple were to be returned.
There’s also concern that allowing a maximum of six publications could allow a small number of superstars to dominate a submission, and a suggestion is that the minimum number moves from 0 to 1, so at least one publication from every member of research active staff is returned. Now this really would cause a rush to move those perceived – rightly or wrongly – as weak links off research contracts! I’m reminded of my MPhil work on John Rawls here, and his work on the difference principle, under which nearly just society seeks to maximise the minimum position in terms of material wealth – to have the richest poorest possible. Would this lead to a renewed focus on support for career young researchers, for those struggling for whatever reason, to attempt to increase the quality of the weakest paper in the submission and have the highest rated lowest rated paper possible?
Or is there any point in doing any of that, when income is only associated with 3 (just) and 4? Do we know how the quality of the ‘tail’ will feed into research income, or into league tables if it’s prestige that counts? I’ll need to think a bit more about this one. My instinct is that I like this idea, but I worry about unintended consequences (“Quick, Professor Fourstar, go and write something – anything – with Dr Career Young!”).
Portability
On portability – whether a researcher’s publications move with them (as previously) or stay with the institution where they were produced (like impact) – the consultation first notes possible issues about what it doesn’t call a “transfer window” round about the REF census date. If you’re going to recruit someone new, the best time to get them is either at the start of a REF cycle or during the meaningless end-of-season games towards the end of the previous one. That way, you get them and their outputs for the whole season. True enough – but hard to see that this is worse than the current situation where someone can be poached in the 89th minute and bring all their outputs with them.
The consultation’s second concern is verification. If someone moves institution, how do we know which institution can claim what? As we found with open access, the point of acceptance isn’t always straightforward to determine, and that’s before we get into forms of output other than journal articles. I suppose my first thought is that point-of-submission might be the right point, as institutional affiliation would have to be provided, but then that’s self declared information.
The consultation document recognises the concern expressed about the disadvantage that portability may have for certain groups – early career researchers and (a group I hadn’t considered) people moving into/out of industry. Two interesting options are proposed – firstly, that publications are portable for anyone on a fixed term contract (though this may inadvertently include some Emeritus Profs) or for anyone who wasn’t returned to REF 2014.
One other non-Stern alternative is proposed – that proportionate publication sharing between old and new employer take place for researchers who move close to the end date. But this seems messy, especially as different institutions may want to claim different papers. For example if Dr Nomad wrote a great publication with co-authors from Old and from New, neither would want it as much as a great publication that she wrote by herself or with co-authors from abroad. This is because both Old and New could still return that publication without Dr Nomad because they had co-authors who could claim that publication, and publications can only be returned once per UoA, but perhaps multiple times by different UoAs.
Overall though – that probable non-starter aside – I’d say portability is happening, and it’s just a case of how to protect career young researchers. And either non-return last time, or fixed term contract = portability seem like good ideas to me.
Interestingly, there’s also a question about whether impact should become portable. It would seem a bit odd to me of impact and publications were to swap over in terms of portability rules, so I don’t see impact becoming portable.
Impact
I’m not going to say too much about impact here and now- this post is already too long, and I suspect someone else will say it better.
Miscellaneous
Other than that…. should ORCID be mandatory? Should Category C (staff not employed by the university, but who research in the UOA) be removed as an eligible category? Should there be a minimum fraction of FTE to be returnable (to prevent overseas superstars being returnable on slivers of contracts)? What exactly is a research assistant anyway? Should a reserve publication be allowed when publication of a returned article is expected horrifyingly close to the census date? Should quant data be used to support assessment in disciplines where it’s deemed appropriate? Why do birds suddenly appear, every time you are near, and what metrics should be used for measuring such birds?
There’s a lot more to say about this, and I’ll be following discussions and debates on twitter with interest. If time allows I’ll return to this post or write some more, less knee-jerky comments over the next days and weeks.
This article first appeared in Funding Insight on 11th May 2017 and is reproduced with kind permission of Research Professional. For more articles like this, visit www.researchprofessional.com