Career Young Researchers | Cash for Questions: Research funding, policy and development

News from the ESRC: International co-investigators and the Future Leaders Scheme

"They don't come over here, they take our co-investigator jobs..." I’m still behind on my blogging – I owe the internet the second part of the impact series, and a book review I really must get round to writing. But I picked up an interesting nugget of information regarding the ESRC and international co-investigators that’s worthy of sharing and commenting upon.

ESRC communications send round an occasional email entitled ‘All the latest from the ESRC’, which is well worth subscribing to, and reading very carefully as often quite big announcements and changes are smuggled out in the small print. In the latest version, for example, the headline news is the Annual Report (2011-12), while the announcement of the ESRC Future Leaders call for 2012 is only the fifth item down a list of funding opportunities. To be fair, it was also announced on Twitter and perhaps elsewhere too, and perhaps the email has a wider audience than people like me. But even so, it’s all a bit low key.

I’ve not got much to add to what I said last year about the Future Leaders Scheme other than to note with interest the lack of an outline stage this year, and the decision to ring fence some of the funding for very early career researchers – current doctoral students and those who have just passed their PhD. Perhaps the ESRC are now more confident in institutions’ ability to regulate their own submission behaviour, and I can see this scheme being a real test of this. I know at the University of Nottingham we’re taking all this very seriously indeed, and grant writing is now neither a sprint nor a marathon but more like a steeplechase, and my impression from the ARMA conference is that we’re far from alone in this. Balancing ‘demand management’ with a desire to encourage applications is a topic for another blog post. As is the effect of all these calls with early Autumn deadlines – I’d argue it’s much harder to demand manage over the summer months when applicants, reviewers, and research managers are likely to be away on holiday and/or researching.

Something else mentioned in the ESRC is a light touch review of the ESRC’s international co-investigator policy. One of the findings was that

“…grant applications with international co-investigators are nearly twice as likely to be successful in responsive mode competitions as those without, strengthening the argument that international cooperation delivers better research.”

This is very interesting indeed. My first reaction is to wonder whether all of that greater success can be explained by higher quality, or whether the extra value for money offered has made a difference. Outside of the various international co-operation/bilateral schemes, the ESRC would generally expect only to pay directly incurred research costs for ICo-Is, such as travel, subsistence, transcription, and research assistance. It won’t normally pay for investigator time and will never pay overheads, which represents a substantial saving on naming a UK-based Co-I.

While the added value for money argument will generally go in favour of the application, there are circumstances where it might make it technically ineligible. When the ESRC abolished the small grants scheme and introduced the floor of £200k as the minimum to be applied for through the research grants scheme, the figure of £200k was considered to represent the minimum scale/scope/ambition that they were prepared to entertain. But a project with a UK Co-I may sneak in just over £200k and be eligible, yet an identical project with an ICo-I would not be eligible as it would not have salary costs or overheads to bump up the cost. I did raise this with the ESRC a while back when I was supporting an application that would be ineligible under the new rules, but we managed to submit it before the final deadline for Small Grants. The issue did not arise for us then, but I’m sure it will (and probably has) arisen for others.

The ESRC has clarified the circumstances under which they will pay overseas co-investigator salary costs:

“….only in circumstances where payment of salaries is absolutely required for the research project to be conducted. For example, where the policy of the International Co-Investigator’s home institution requires researchers to obtain funding for their salaries for time spent on externally-funded research projects.

In instances where the research funding structure of the collaborating country is such that national research funding organisations equivalent to the ESRC do not normally provide salary costs, these costs will not be considered. Alternative arrangements to secure researcher time, such as teaching replacement costs, will be considered where these are required by the co-investigator’s home institution.”

This all seems fairly sensible, and would allow the participation of researchers involved in Institutes where they’re expected to bring in their own salary, and those where there isn’t a substantial research time allocation that could be straightforwardly used for the project.

While it would clearly be inadvisable to add on an ICo-I in the hope of boosting chances of success or for value for money alone, it’s good to know that applications with ICo-Is are doing well with the ESRC even outside of the formal collaborative schemes, and that we shouldn’t shy away from looking abroad for the very best people to work with. Few would argue with the ESRC’s contention that

[m]any major issues requiring research evidence (eg the global economic crisis, climate change, security etc.) are international in scope, and therefore must be addressed with a global research response.

How can we help researchers get responses for web questionnaires?

A picture of an energy saving lightbulb — *Insert your own hilarious and inaccurate joke about how long energy saving lightbulbs take to warm up here*

I’ve had an idea, and I’d like you, the internet, to tell me if it’s a good one or not. Or how it might be made into a good one.

Would it be useful to set up a central list/blog/twitter account for ongoing research projects (including student projects) which need responses to an internet questionnaire from the general public? Would researchers use it? Would it add value? Would people participate?

Every so often, I receive an email or tweet asking for people to complete a research questionnaire on a particular topic. I usually do (if it’s not too long, and the topic isn’t one I consider intrusive), partly because some of them are quite interesting, partly because I feel a general duty to assist with research when asked, and partly because I probably need to get out more. The latest one was one which a friend and former colleague shared on Facebook. It was a PhD project from a student in her department about sun tanning knowledge and behaviour, and it’s here if you feel like taking part. Now this is not a subject that I feel passionately about, but perhaps that’s why it might be useful for the likes of me to respond.

I guess the key assumptions that I’m making are that there are sufficient numbers of other people like me who would be willing to spend a few minutes every so often completing a web survey to support research, and that nothing like this exists already. If there is, I’ve not heard about and I’d have thought that I would have done. But I’d rather be embarrassed now rather than later! Another assumption is that such a resource might be useful. I strongly suspect that any such resource would have a deeply atypical demographic – I’d imagine it would be mainly university staff and students. But I’d imagine that well designed research questionnaires would be asking sufficiently detailed demographic information to be able to factor this in. For some student projects where the main challenge can be quantity rather than variety, this might not even matter too much. I guess it depends what questions are being asked as part of the research.

I’ve not really thought this through at all yet. I would imagine that only projects which could be completed by anyone would be suitable for inclusion, or at least only projects where responses are invited from a broad range of people. Very specific projects probably wouldn’t work, and would make it harder for participants to find ones which they can do. Obviously all projects would need to have ethical approval from their institution. There would be an expectation that beneficiaries are prepared to reciprocate and help others in return. And clearly there has to be a strategy to tell people about it.

In practical terms, I’m thinking about either a separate blog or a separate page of this one, and probably a separate twitter account. Researchers could add details in a comment on a monthly blog post, and either tweet the account and ask for a re-tweet, or email me a tweet to send. Participants could follow the twitter feed and subscribe to the comments and blog.

So… what do you think? Please comment below (or email me if you prefer). Would this be useful? Would you participate? What have I missed? If I do set this up, how might I go about telling people about it?

Coping with rejection: What to do if your grant application is unsuccessful. Part 1: Understand what it means…. and what it doesn’t mean

You can't have any research funding. In this life, or the next....

Some application and assessment processes are for limited goods, and some are for unlimited goods, and it’s important to understand the difference. PhD vivas and driving tests are assessments for unlimited goods – there’s no limit on how many PhDs or driving licenses can be issued. In principle, everyone could have one if they met the requirements. You’re not going to fail your driving test because there are better drivers than you. Other processes are for limited goods – there is (usually) only one job vacancy that you’re all competing for, only so many papers that a top journal accept, and only so much grant money available.

You’d think this was a fairly obvious point to make. But talking to researchers who have been unsuccessful with a particular application, there’s sometimes more than a hint of hurt in their voices as they discuss it, and talk in terms of their research being rejected, or not being judged good enough. They end up taking it rather personally. And given the amount of time and effort that must researchers put into their applications, that’s not surprising.

It reminds me of an unsuccessful job applicant whose opening gambit at a feedback meeting was to ask me why I didn’t think that she was good enough to do the job. Well, my answer was that I was very confident that she could do the job, it’s just that there was someone more qualified and only one post to fill. In this case, the unsuccessful applicant was simply unlucky – an exceptional applicant was offered the job, and nothing she could have said or done (short of assassination) would have made much difference. While I couldn’t give the applicant the job she wanted or make the disappointment go away, I could at least pass on the panel’s unanimous verdict on her appointability. My impression was that this restored some lost confidence, and did something to salve the hurt and disappointment. You did the best that you could. With better luck you’ll get the next one.

Of course, with grant applications, the chances are that you won’t get to speak to the chair of the panel who will explain the decision. You’ll either get a letter with the decision and something about how oversubscribed the scheme was and how hard the decisions were, which might or might not be true. Your application might have missed out by a fraction, or been one of the first into the discard pile.

Some funders, like the ESRC, will pass on anonymised referees’ comments, but oddly, this isn’t always constructive and can even damage confidence in the quality of the peer review process. In my experience, every batch of referees’ comments will contain at least one weird, wrong-headed, careless, or downright bizarre comment, and sometimes several. Perhaps a claim about the current state of knowledge that’s just plain wrong, a misunderstanding that can only come from not reading the application properly, and/or criticising it on the spurious grounds of not being the project that they would have done. These apples are fine as far as they go, but they should really taste of oranges. I like oranges.

Don’t get me wrong – most referees’ reports that I see are careful, conscientious, and insightful, but it’s those misconceived criticisms that unsuccessful applicants will remember. Even ahead of the valid ones. And sometimes they will conclude that its those wrong criticisms that are the reason for not getting funded. Everything else was positive, so that one negative review must be the reason, yes? Well, maybe not. It’s also possible that that bizarre comment was discounted by the panel too, and the reason that your project wasn’t funded was simply that the money ran out before they reached your project. But we don’t know. I really, really, really want to believe that that’s the case when referees write that a project is “too expensive” without explaining how or why. I hope the panel read our carefully constructed budget and our detailed justification for resources and treat that comment with the fECing contempt that it deserves.

Fortunately, the ESRC have announced changes to procedures which allow not only a right of reply to referees, but also to communicate the final grade awarded. This should give a much stronger indication of whether it was a near miss or miles off. Of course, the news that an application was miles off the required standard may come gifted wrapped with sanctions. So it’s not all good news.

But this is where we should be heading with feedback. Funders shouldn’t be shy about saying that the application was a no-hoper, and they should be giving as much detail as possible. Not so long ago, I was copied into a lovely rejection letter, if there’s any such thing. It passed on comments, included some platitudes, but also told the applicant what the overall ranking was (very close, but no cigar) and how many applications there were (many more than the team expected). Now at least one of the comments was surprising, but we know the application was taken seriously and given a thorough review. And that’s something….

So… in conclusion…. just because your project wasn’t funded doesn’t (necessarily) mean that it wasn’t fundable. And don’t take it personally. It’s not personal. Just the business of research funding.

Estimating Investigator and Researcher Time on a Project

Prompted in part by an interesting discussion of the importance of the budget in establishing the overall credibility and shape of a research proposal at the ever-excellent Research Whisperer, I thought I’d put fingers-to-keyboard on the vexed issue of estimating staff time on research grant applications. Partly this is to share some of what I do and what I recommend, but mostly it’s to ask others how they approach it. Comments, thoughts, suggestions, and experiences welcome as ever in the comments below.

Estimating staff time is by some distance the hardest part of the budget, and often when I discuss this with academic colleagues, we end up in a kind of “I dunno, what do you reckon?” impasse. (I should say that it’s me who speaks like that, not them). I’ve never been involved in a research project (other than my ‘desk research’ MPhil), so I’ve really no idea, and I don’t have any particular insight into how long certain tasks will take. Career young academics, and the increasing number of academics who have never had the opportunity to lead on a research project, have little experience to draw upon, and even those who have experience seem to find it difficult. I can understand that, because I’d find it difficult too. If someone where to ask me how estimate how much time I spent over the course of a year on (say) duties related to my role as School Research Ethics Officer, I’d struggle to give them in answer in terms of a percentage FTE or in terms of a total number of working days.

It’s further complicated by the convenient fiction that academic staff work the standard five day weeks, and 37.5 hour working weeks. Even those who don’t regularly work evenings and weekends will almost certainly have flexible working patterns that make it even harder to estimate how long something will take. This means that the standard units of staff time that are most often used – total days and percentage of full time – aren’t straightforwardly convertible from one currency to the other. To make matters worse, some European funding sources seem to prefer ‘person months’. Rather than the standard working week, I think this probably reflects the reality of academic work in many institutions:

The response to my question about how much time the project will take is often answered with another question. What will the funder pay for? What looks right? What feels right? Longer, thinner project, or shorter and more intensive? The answer is always and inevitably, ‘it depends’. It depends upon what is right for the project. A longer less intensive project might make sense if you have to wait for survey responses to come in, or for other things to happen. On the other hand, if it’s something that you can and want to work on intensively, go for it. But the project has to come first. What would it take to properly take on this research challenge?

Often the answer is a bit hand-wavy and guesstimate-ish. Oh, I don’t know….say… about, two days per week? Two and a half? Would they fund three? This is a generally a good starting point. What’s not a good starting point is the kind of fifteen-minutes-every-Tuesday approach, where you end up with a ‘salami project’ – a few people involved, but sliced so thinly it’s hard to see how anything will get done. This just isn’t credible, and it’s very unlikely to get funded by the likes of the ESRC (and other funders too, I’m sure) if they don’t believe that you’ll possibly be able to deliver for the amount of money (time) you’re asking for. Either they’ll conclude that you don’t understand what your own research requires (which is fatal for any chance of funding), or they’ll think that you’re trying to pull a fast one in terms of the value for money criterion. And they won’t like that much either.

The other way to make sure that you’re not going to get funded is to go the other way and become greedy. I’ve noticed that – oddly – academics seldom over-estimate their own time, but over-estimate the amount of researcher time required. I’ve seen potential bids before now that include a heavy smattering of research associates, but no clear idea what it actually is they’ll be doing all day, other than doing stuff that the lead investigator doesn’t want to do. In a UK context, overheads are calculated on the basis of investigator and researcher time, so including researchers is doubly expensive. One way round this that I often recommend is to ensure that what’s required is actually a research associate (who attracts overheads) rather than an academic-related project manager or administrator (who doesn’t). But if it’s hard to estimate how long it will take you to do any given task, it’s doubly hard to estimate how long it will take someone else – usually someone less experienced and less skilled – and perhaps from a cognate sub-discipline rather than from your own.

My usual advice is for would-be principal investigators to draw up a table showing the various phases of the project as rows, and project staff as columns. In addition to the main phases of the research, extra rows should be added for the various stages of dissemination and impact activity; for project coordination with colleagues; for line management of any researchers or administrative/managerial staff; for professional development where appropriate. Don’t forget travelling time associated with meetings, fieldwork, conferences etc. The main thing that I usually see underestimated is project management time. As principal investigator, you will have to meet reasonably regularly with your finance people, and you’ll have to manage and direct the project research associates. Far too many academics seem to see RAs as clones who will instinctively know what to do and don’t need much in the way of direction, advice, or feedback.

Once this is done, I suggest adding up the columns and then working backwards from those total numbers of project days to a percentage FTE, or days per week, or whatever alternative metric you prefer. In the UK, the research councils assuming that 220 days = 1 working year, so you can use this to calculate the percentage of full time. The number that you come out with should feel intuitively about right. If it doesn’t, then something has probably gone wrong. Go back to the table, and adjust things a little. By going to and fro, from the precision of the table to the intuition of the percentage of time, you should reach what my philosophical hero and subject of my thesis, John Rawls, called ‘reflective equilibrium’. Though he wasn’t talking about investigator time. Once you’re happy with the result, you should probably use the figure from the table, and you should certainly consider putting the table in full in the ‘justification for resources’ section.

Something I’m starting at the moment as part of the end-of-project review process is to go back to the original estimates of staff time, and to get a sense from the research team about how accurate they were, and what they would estimate differently next time, if anything. The two things that have come out strongly from this so far I’ve outlined above – managing staff and project administration – but I’ll be looking for others.

So…. over to you. How do you estimate researcher and investigator time? Have you been involved in a funded project? If so, what did you miss in your forecasts (if anything)? What would you do differently next time?

What would wholesale academic adoption of social media look like?

Last week, the LSE Impact of Social Sciences blog was asking for its readers and followers to nominate their favourite academic tweeters. This got me thinking. While that’s a sensible question to ask now, and one that could create a valuable resource, I wonder whether the question would make as much sense if asked in a few years time?

The drivers for academics (and arguably academic-related types like me) to start to use twitter and to contribute to a blog are many – brute self-promotion; desire to join a community or communities; to share ideas; to test ideas; to network and make new contacts; to satisfy the impact requirements of the research funder; and so on and so forth. I think most current PhD students would be very well advised to take advantage of social media to start building themselves an online presence as an early investment in their search for a job (academic or otherwise). I’d imagine that a social media strategy is now all-but-standard in most ESRC ‘Pathways to Impact’ documents. Additionally, there are now many senior, credible, well-established academic bloggers and twitterers, many of whom are also advocates for the use of social media.

So, what would happen if there was a huge upsurge in the number of academics (and academic-relateds) using social media? What if, say, participation rates reach about 20% or so? Would the utility of social media scale, or would the noise to signal ratio be such that its usefulness would decrease?

This isn’t a rhetorical question – I’ve really no idea and I’m curious. Anyone? Any thoughts?

I guess that there’s a difference between different types of social media. I have friends who are outside the academy and who have Twitter accounts for following and listening, rather than for leading or talking. They follow the Brookers and the Frys and the Goldacres, and perhaps some news sources. They use Twitter like a form of RSS feed, essentially.

But what about blogging, or using Twitter to transmit, rather than to receive? If even 10% of academics have an active blog, will it still be possible or practical to keep track of everything relevant that’s written. In my field, I think I’ve linked to pretty much every related blog (see links in the sidebar) in the UK, and one from Australia. In certain academic fields it’s probably similarly straightforward to keep track of everyone significant and relevant. If this blogging lark catches on, there will come a point at which it’s no longer possible for anyone to keep up with everything in any given field. So, maybe people become more selective and we drop down to sub-specialisms, and it becomes sensible to ask for our favourite academic tweeters on non-linear economics, or something like that.

On the other hand, it might be that new entrants to the blogging market will be limited and inhibited by the number already present. Or we might see more multi-author blogs, mergers etc and so on until we re-invent the journal. Or strategies that involve attracting the attention and comment of influential bloggers and the academic twitterati (a little bit of me died inside typing that, I hope you’re happy….). Might that be what happens? That e-hierarchies form (arguably they already exist) that echo real world hierarchies, and effectively squeeze out new entrants? Although… I guess good content will always have a chance of ‘going viral’ within relevant communities.

Of course, it may well be that something else will happen. That Twitter will end up in the same pile as MySpace. Or that it simply won’t be widely adopted or become mainstream at all. After all, most academics still don’t have much of a web 1.0 presence beyond a perfunctory page on their Department website.

That’s all a bit rambly and far longer than I meant it to be. But as someone who is going to be recommending the greater use of social media to researchers, I’d like to have a sense of where all this might be going, and what the future might hold. Would the usefulness of social media as an academic communication, information sharing, and networking tool effectively start to diminish once a certain point is reached? Or would it scale?