{"id":1389,"date":"2024-05-17T14:18:29","date_gmt":"2024-05-17T13:18:29","guid":{"rendered":"https:\/\/socialscienceresearchfunding.co.uk\/?p=1389"},"modified":"2024-05-17T14:34:00","modified_gmt":"2024-05-17T13:34:00","slug":"the-art-of-the-sift","status":"publish","type":"post","link":"https:\/\/socialscienceresearchfunding.co.uk\/?p=1389","title":{"rendered":"The art of the sift"},"content":{"rendered":"\n

A version of this article first appeared in Funding Insight in February 2022 and is reproduced with kind permission of Research Professional. For more articles like this, visit <\/em>www.researchprofessional.com<\/em><\/a><\/p>\n\n\n\n

How to select bids when funders restrict the number that each university can submit<\/strong><\/p>\n\n\n\n

\"\"
Nicolas Poussin (1594\u20131665), The Judgment of Solomon (1649), oil on canvas, 101 x 150 cm, Mus\u00e9e du Louvre, Paris. Wikimedia Commons.<\/figcaption><\/figure>\n\n\n\n

One of the most awkward challenges in research development is responding to a \u2018restricted\u2019 funding call that only permits a limited number of applications per university. This requires an internal selection process. I\u2019m going to share some of the things I do when I set one up. I don\u2019t have all the answers, and I\u2019d be interested to hear what others do, via twitter, or email or the comments.<\/p>\n\n\n\n

This article refers primarily to funding schemes with a hard limit on application numbers. In the UK, that includes the Leverhulme Trust\u2019s major calls and the Academy of Medical Science\u2019s Springboard Awards. Some of the suggestions may also be relevant to panels for schemes with a \u2018soft\u2019 limit. These typically don\u2019t set a formal limit on application numbers, but require universities to have a process to manage demand, submit only their most competitive applications, and not support others. There are good arguments for saying that we should be doing this sift anyway, if only to prevent our researchers wasting time and effort.<\/p>\n\n\n\n\n\n\n\n

Don\u2019t shy away from considering radical options. <\/strong>The rest of this article is going to assume that we want a bottom-up, open call approach to selecting the university\u2019s candidates. But a university could<\/em> adopt a top-down approach and pick winners instead. No burdensome selection processes, just choose your candidate and give them a clear run.  I\u2019m going to further assume that we even want<\/em> to participate in schemes which require a complex sift process. We could<\/em> decide that some smaller and\/or low success rate schemes aren\u2019t worth our time and effort. Or we could decide our candidate for lower value awards using a lottery.<\/p>\n\n\n\n

But I think a bottom-up approach is more likely to be successful in finding strong candidates and proposals, especially for major calls, and is a better fit for an open, democratic university with a positive research culture. So that\u2019s the approach I\u2019m going with.<\/p>\n\n\n\n

Start the process before the call is out.<\/strong> However much time the funder allows between official call launch and deadline, it\u2019s probably not enough. With a restricted call, you need: time for applicants to hone their ideas; time to run a selection process; and time to support the development of the chosen proposal. <\/p>\n\n\n\n

Offer informal advice to applicants. <\/strong>Avoid wasted effort by having someone\u2014perhaps a research development manager in a central team\u2014serving as call contact point. Anyone who\u2019s interested should talk through their ideas and get feedback about their suitability in terms of fit-to-scheme and internal competitiveness. Encouraging those who need encouraging and redirecting the energies of those whose research doesn\u2019t fit this opportunity is time well spent.<\/p>\n\n\n\n

Don\u2019t decide too late.<\/strong> Most applicants, perfectly reasonably, will only put in a finite amount of effort into their proposals in advance of the internal selection process. Make sure that there\u2019s plenty of time for onward development and further rounds of review and feedback after your candidate has been selected. If you publicise the call and the likely timeline and offer informal advice in advance of the formal launch, you give yourself license to have an earlier deadline and therefore an earlier decision. After all, potential applicants have been given plenty of notice, and should be working up their ideas in preparation.<\/p>\n\n\n\n

Mirror the call\u2026 in miniature. <\/strong>The internal review documentation should resemble a cut-down version of the funder\u2019s application form. But pick out only the key sections and ask applicants to submit only those sections that will help an internal panel reach a decision. If there are long \u2018case for support\u2019 sections, reduce the page limit and concentrate on key elements. If there are sections on career history and publications, ask for a short, targeted CV. Only interview if the funder does as part of the selection process. Ask for an outline costing to ensure the bid fits the budget limits.<\/p>\n\n\n\n

Select your panel members<\/strong>. It\u2019s important to do this early so you can get a date in the diary. Try to get senior colleagues who are broadly representative of the disciplines that are likely to apply. Look for previous award winners, and those with experience of the funder. The aim is to have a balanced panel that has the right mix of expertise and will be regarded as fair by applicants.<\/p>\n\n\n\n

It\u2019s a good idea to look to your senior university research leadership for panel members. This sort of service ought to be part of their role. But I\u2019ve also seen early\/mid-career researchers make outstanding<\/em> review panel contributions. If it\u2019s a new responsibility, they tend to take it very seriously and it\u2019s a good developmental opportunity to see how a review panel works from the inside.<\/p>\n\n\n\n

Transparency about panel membership is an interesting issue. I\u2019ve found that potential applicants want to satisfy themselves that their research will get a fair hearing. I\u2019ll normally describe panel members in terms of jobs\/role titles: senior researcher in School X or director of research Faculty Y for example, but without naming names. It\u2019s not fully transparent, but it\u2019s not opaque. Applicants want a sense of what kinds of reviewers and what roles they\u2019ll play, and that\u2019s often enough to satisfy some of their inherent suspicion of any \u2018central\u2019 panel outside their own field.<\/p>\n\n\n\n

Prime your panel. <\/strong>I\u2019ve run short briefing sessions for panel members in advance of them having sight of proposals. I run through the funder and the scheme and what they\u2019re looking for so that we\u2019re all looking for the same things. I usually suggest one criterion: which proposal is most likely to be successful, given what we know of the funder and the scheme. This primes members to think in terms of competitiveness, not just their own personal view of merit.<\/p>\n\n\n\n

Similarly, reviewers should be told to assume that fixable faults are fixed and that they are reviewing the best possible iteration of each proposal, given the limited time available for further development before the deadline. Be clear to panel members about their role, which should be to bring their given expertise to the table, not to be an advocate at the panel for School X.<\/p>\n\n\n\n

Structure your panel meetings like funder panel meetings. <\/strong>Don\u2019t leave your selection panel to talk through proposals one at a time. They\u2019ll spend ages on the first few and then run out of time. I ask panel members to read and rank each application independently in advance and send me the rankings to collate. If you use Microsoft Forms to collect responses, it\u2019ll do this automatically and generate neat infographics. Though it defaults to a linear weighting system for rankings (i.e. 3 points for first place, 2 for second, 1 for third) – you might prefer a greater points weighting for higher ranked proposals.<\/p>\n\n\n\n

I share these rankings at the start of the meeting, putting the proposals that need most discussion at the top of the agenda. If we can only submit one proposal, there\u2019s no point spending ages discussing the proposal that nobody ranked higher than third. If we can submit three or four bids and have unanimity about the best two, we should focus on the three others jostling for the remaining slots.<\/p>\n\n\n\n

Independent rankings give equal weight to the views of all reviewers<\/strong>. It places all the cards on the table and shows the mood of the meeting straight away. It makes it harder for any one panel member to aggressively champion their preferred proposal if we have a clear ranking showing a lack of broader support. Of course, reviewers may change their ranking after discussion, but rankings show a clear direction of initial views which focuses minds and discussion.<\/p>\n\n\n\n

Pre-nominate one reviewer to \u2018introduce\u2019 each proposal and another to \u2018second\u2019<\/strong>. They\u2019re responsible for leading on feedback and summarising the proposal for the panel.<\/p>\n\n\n\n

Take extra care with feedback to unsuccessful applicants<\/strong>. As this is internal, this is extra sensitive. Institutionally, you have an ongoing duty to support these researchers. The first line of feedback should just be that the panel thought other proposals had a better chance of success, and that has as much to do with the proposal(s) that were chosen as those that were not. Appropriately edited reviewer comments should be shared, but in the context of ‘this is what reviewers thought of your application’ not ‘this is why we didn’t pick your bid’. The aim should be to support development, and where appropriate, identify other funding opportunities.<\/p>\n\n\n\n

Meet successful applicants<\/strong>. I\u2019d usually send written comments and then try to arrange a meeting with a subsection of the review panel. The aim of that meeting is to elaborate on the written feedback and to give the applicant(s) an opportunity to respond and ask questions, to further develop their proposal. This is a long list, I realise. But take comfort in the fact that once you have concluded this process and your decision is made, you have nothing more to do. That is, apart from all the usual work that goes into supporting, developing and submitting a competitive grant application.<\/p>\n\n\n\n

<\/p>\n","protected":false},"excerpt":{"rendered":"

A version of this article first appeared in Funding Insight in February 2022 and is reproduced with kind permission of Research Professional. For more articles like this, visit www.researchprofessional.com How to select bids when funders restrict the number that each university can submit One of the most awkward challenges in research development is responding to … Continue reading “The art of the sift”<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_post_was_ever_published":false},"categories":[8,10,6,13,19],"tags":[34,37,35,28,29],"class_list":["post-1389","post","type-post","status-publish","format-standard","hentry","category-career-young-researchers","category-frustrations","category-funding","category-funding-policy","category-university-culture","tag-demand-management","tag-internal-peer-review","tag-internal-sift","tag-research","tag-research-funding"],"aioseo_notices":[],"aioseo_head":"\n\t\t\n\t\n\t\n\t\n\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t