The ESRC and “Demand Management”: Part 4 – Quotas and Sanctions, PIs and Co-Is….

Previously in this series of posts on ESRC Demand Management I’ve discussed the background to the current unsustainable situation and aspects of the initial changes, such as the greater use of sifting and outline stages, and the new ban on (uninvited) resubmissions.  In this post I’ll be looking forward to the possible measures that might be introduced in a year or so’s time should application numbers not drop substantially….

When the ESRC put their proposals out to consultation, there were four basic strategies proposed.

• Charging for applications
• Quotas for numbers of applications per institution
• Sanctions for institutions
• Sanctions for individual researchers

Reading in between the lines of the demand management section of the presentation that the ERSC toured the country with in the spring, charging for applications is a non-starter.  Even in the consultation documents, this option only appeared to be included for the sake of completeness – it was readily admitted that there was no evidence that it would have the desired effect.

I think we can also all-but-discount quotas as an option.  The advantage of quotas is that it would allow the ESRC to precisely control the maximum number of applications that could be submitted.  Problem is, it’s the nuclear option, and I think it would be sensible to try less radical options first.  If their call for better self-regulation and internal peer review within institutions fails, and then sanctions schemes are tried and fail, then (and only then) should they be thinking about quotas.  Sanctions (and the threat of sanctions) are a seek to modify application submission behaviour, while quotas pretty much dictate it.  There may yet be a time when Quotas are necessary, though I really hope not.

What’s wrong with Quotas, then?  Well, there will be difficulties in assigning quotas fairly to institutions, in spite of complex plans for banding and ‘promotion’ and ‘relegation’ from the bands.  That’ll lead to a lot of game playing, and it’s also likely that there will be a lot of mucking around with the lead applicant.  If one of my colleagues has a brilliant idea and we’re out of Quota, well, maybe we’ll find someone at an institution that isn’t and ask them to lead.  I can imagine a lot of bickering over who should spend their quota on submitting an application with a genuinely 50-50 institutional split.

But my main worry is that institutions are not good at comparing applications from different disciplines.  If we have applications from (say) Management and Law vying for the last precious quota slot, how is the institution to choose between them?  Even if it has experts who are not on the project team, they will inevitably have a conflict of interest – there would be a worry that they would support their ‘team’.  We could give it a pretty good cognate discipline review, but I’m not confident we would always get the decision right.  It won’t take long before institutions start teaming up to provide external preliminary peer review of each other’s applications, and before you know it, we end up just shifting the burden from post-submission to pre-submission for very little gain.

In short, I think quotas are a last resort idea, and shouldn’t be seriously considered unless we end up in a situation where a combination of (a) the failure of other demand management measures, and/or (b) significant cuts in the amount of funding available.

Which leaves sanctions – either on individual researchers or on their institutions.  The EPSRC has had a policy of researcher sanctions for some time, and that’s had quite a considerable effect.  I don’t think it’s so much through sanctioning people and taking them out of the system so much as a kind of chill or placebo effect, whereby greater self-selection is taking place.  Once there’s a penalty for throwing in applications and hoping that some stick, people will stop.

As I argued previously, I think a lot of that pressure for increased submissions is down to institutions rather than individuals, who in many cases are either following direct instructions and expectations, or at least a very strong steer.  As a result, I was initially in favour of a hybrid system of sanctions where both individual researchers and institutions could potentially be sanctioned.  Both bear a responsibility for the application, and both are expected to put their name to it.  But after discussions internally, I’ve been persuaded that individual sanctions are the way to go, in order to have a consistent approach with the EPSRC, and with the other Research Councils, who I think are very likely to have their own version.  While the formulae may vary according to application profiles, as much of a common approach as possible should be adopted, unless of course there are overwhelming reasons why one of the RCs that I’m less familiar with should be different.

For me, the big issue is not whether we end up with individual, institutional, or hybrid sanctions, but whether the ESRC go ahead with plans to penalise co-investigators (and/or their institutions) as well as PIs in cases where an application does not reach the required standard.

This is a terrible, terrible, terrible idea and I would urge them to drop it.  The EPSRC don’t do it, and it’s not clear why the ESRC want to.  For me, the co-I issue is more important than which sanction model we end up with.

Most of the ESRC’s documents on demand management are thoughtful and thorough.  They’re written to inform the consultation exercise rather than dictate a solution, and I think the author(s) should be – on the whole – congratulated on their work.  Clearly a lot of hard work has gone into the proposals, which given the seriousness of the proposals is only right.  However, nowhere is there to be found any kind of argument or justification that I can find for why co-investigators (insert your own ‘and/or institutions’ from here on)  should be regarded as equally culpable.

I guess the argument (which the ESRC doesn’t make) might be that an application will be given yet more careful consideration if more than the principal investigator has something to lose.  At the moment, I don’t do a great deal if an application is led from elsewhere – I offer my services, and sometimes that offer is taken up, sometimes it isn’t.  But no doubt I’d be more forceful in my ‘offer’ if a colleague or my university could end up with a sanctions strike against us.  Further, I’d probably be recommending that none of my academic colleagues get involved in an application without it going through our own rigorous internal peer review processes.  Similarly, I’d imagine that academics would be much more careful about what they allowed their name to be put to, and would presumably take a more active role in drafting the application.  Both institutions and individual academics, can, I think, be guilty of regarding an application led from elsewhere as being a free roll of the dice.  But we’re taking action on this – or at least I am.

The problem is that these benefits are achieved (if they are achieved at all) at the cost of abandoning basic fairness.  It’s just not clear to me why an individual/institution with only a minor role in a major project should be subject to the same penalty as the principal investigator and/or the institution that failed to spot that the application was unfundable.  It’s not clear to me why the career-young academic named as co-I on a much more senior colleague’s proposal should be held responsible for its poor quality.  I understand that there’s a term in aviation – cockpit gradient – which refers to the difference in seniority between Pilot and Co-Pilot.  A very senior Pilot and a very junior co-Pilot is a bad mix because the junior will be reluctant to challenge the senior.  I don’t understand why someone named as co-I for an advisory role – on methodology perhaps, or for a discrete task, should bear the same responsibility.  And so on and so forth.  One response might be to create a new category of research team member less responsible than a ‘co-investigator’ but more involved in the project direction (or part of the project direction) than a ‘researcher’, but do we really want to go down the road of redefining categories?

Now granted, there are proposals where the PI is primus inter pares among a team of equally engaged and responsible investigators, where there is no single, obvious candidate for the role of PI.  In those circumstances, we might think it would be fair for all of them to pay the penalty. But I wonder what proportion of applications are like this, with genuine joint leadership?  Even in such cases, every one of those joint leaders ought to be happy to be named as PI, because they’ve all had equal input.  But the unfairness inherent in only one person getting a strike against their name (and other(s) not), is surely much less unfair than the examples above?

As projects become larger, with £200k (very roughly, between two and two and a half person-years including overheads and project expenses) now being the minimum, the complex, multi-armed, innovative, interdisciplinary project is likely to be come more and more common, because that’s what the ESRC says that it wants to fund.   But the threat of a potential sanction (or step towards sanction) for every last co-I involved is going to be a) a massive disincentive to large-scale collaboration, b) a logistical and organisational nightmare, or c) both.

Institutionally, it makes things very difficult.  Do we insist that every last application involving one of our academics goes through our peer review processes?  Or do we trust the lead institution?  Or do we trust some (University of Russell) but not others (Poppleton University)?  How does the PI manage writing and guiding the project through various different approval processes, with the danger that team members may withdraw (or be forced to withdraw) by their institution?  I’d like to think that in the event of sanctions on co-Is and/or institutions that most Research Offices would come up with some sensible proposals for managing the risk of junior-partnerdom in a proportionate manner, but it only takes one or two to start demanding to see everything and to run everything to their timetable to make things very difficult indeed.