Reviewing Internal Review of Grant Applications (part 1): Helping or Hoop-jumping?

A version of this article first appeared in Funding Insight in April 2019 and is reproduced with kind permission of Research Professional. For more articles like this, visit www.researchprofessional.com

Most universities have internal peer review processes for research grant applications. In the first of two articles about internal peer review, I wonder whether what ought to be valuable support can be perceived as an obstacle. Part two looks at how we might run peer review more effectively.

Why do we have internal peer review?

Internal peer review of research grant applications has two distinct functions which can easily become blurred. I think this can cause misunderstandings.

The first function is as filter – to select which applications go forward and which do not. This has two variants. A ‘hard filter’ for a scheme or funder with formal limits on the number of applications that one institution can submit. Or a ‘soft filter’ where there are no formal limits on application numbers, but there’s a steer from the funder to submit only the most competitive applications. Another motivation for a soft filter is to save academic time by slowing, stopping, or redirecting uncompetitive applications.

The second function is to improve the quality of the application. The goal is to produce some actionable suggestions for improvements to increase the chance of success. In a previous article I explained how research development staff can bring a fresh perspective. Comments from a senior academic of comparable standing to the expert reviewers or funding panel members can be similarly helpful, but with the added benefit of academic expertise.

Both functions of peer review – of filtering and improving – are often rolled together into one process. Perhaps this causes confusion both for reviewers and the reviewed. I wonder if we over-emphasise the role of the filter at the expense of the improvement? Does fear of the filter reduce the efficacy of the suggestions for improvements?

Perceptions of internal peer review

When discussing internal peer review with academic colleagues, I’ve seen wildly different reactions. Some are very enthusiastic and are hungry for comments and feedback. Others are a bit more…. Gollum and don’t want anyone to gaze upon their precious. Most are somewhere in the middle… welcoming of genuinely useful comments and insights, but wary about being forced to make changes against their better judgement or being prevented from applying at all.

There’s no denying that ‘filter’ role exists and it would be mistake to do so. I reassure academics that in my experience it’s very rare for a bid to be soft-filtered out because of internal reviewer’s comments and for the applicant to disagree with the rationale. Usually the reviewer has spotted something that the applicant missed, either related to the call, the application or underpinning idea. Perhaps it needs another co-I, or needs stronger stakeholder engagement, needs to engage with a particular body of literature, or just needs a lot more time to develop. Or the issue is the fit to funder or call.

Research development staff send out details of calls with internal timetables and internal deadlines for the various review stages. But are potential applicants seeing peer review (and associated deadlines) as a developmental process put in place to support them and to help them succeed? Or do they see peer review as a barrier to be overcome (or even evaded), placed in their path by over-officious Heads of Research and research managers that seek to micromanage, police and restrict?

I sometimes worry that in our desire to set out processes to try to prevent and pre-empt disruptive last-minute applications and set out an orderly and timely process, we end up sending the wrong message about peer review and about the broader support available. If we’re dictating terms and timetables for peer review, do we make it look as if grant applicants must fit around reviewer (and research support) requirements and timescales? And is that the right way around?

To be clear, I’m certainly not arguing against having a structured process with indicative milestones with some level of enforcement. Unplanned last minute applications are disruptive and stressful, forcing people to drop everything to provide support with no notice. Worst of all, the applications that result usually aren’t very good and rushed applications are seldom competitive. We absolutely should try to save people from this kind of folly.

And… of course, we need to allow time for senior (and therefore busy) academics to undertake internal peer review. I suspect that most institutions rely on a relatively small pool of reviewers who are asked to read and comment on multiple applications per year, and that few get any formal workload allocation. While we should certainly give applicants plenty of time to write their applications, we need to treat our reviewers with consideration and value their time.

Positive about internal peer review

I’m not arguing that we disguise or minimise the ‘filter’ element of internal peer review in favour of an unqualified upbeat presentation of internal peer review being entirely about improving the quality of the application. But perhaps we could look at ways to present internal peer review in a more positive, supportive, developmental – and less officious – light.

The most important part of peer review positivity – and the subject of the second part of this series – is in how internal peer review happens in practice: who reviews, how and when; and how and in what spirit reviewer comments are communicated to applicants. If internal peer review as a process helps strengthen applications, word will get round and support and buy-in will grow – one positive experience at a time.

But even before that stage, I think it’s worth thinking about how we communicate our internal peer review processes and timetables. Could we be more positive in our framing and communication? Could we present internal peer review more as a helping hand to climb higher, and less as a hurdle to overcome?