Estimating Investigator and Researcher Time on a Project

PS: Time is also overheads

Prompted in part by an interesting discussion of the importance of the budget in establishing the overall credibility and shape of a research proposal at the ever-excellent Research Whisperer, I thought I’d put fingers-to-keyboard on the vexed issue of estimating staff time on research grant applications.  Partly this is to share some of what I do and what I recommend, but mostly it’s to ask others how they approach it. Comments, thoughts, suggestions, and experiences welcome as ever in the comments below.

Estimating staff time is by some distance the hardest part of the budget, and often when I discuss this with academic colleagues, we end up in a kind of “I dunno, what do you reckon?” impasse.  (I should say that it’s me who speaks like that, not them).  I’ve never been involved in a research project (other than my ‘desk research’ MPhil), so I’ve really no idea, and I don’t have any particular insight into how long certain tasks will take.  Career young academics, and the increasing number of academics who have never had the opportunity to lead on a research project, have little experience to draw upon, and even those who have experience seem to find it difficult.  I can understand that, because I’d find it difficult too.  If someone where to ask me how estimate how much time I spent over the course of a year on (say) duties related to my role as School Research Ethics Officer, I’d struggle to give them in answer in terms of a percentage FTE or in terms of a total number of working days.

It’s further complicated by the convenient fiction that academic staff work the standard five day weeks, and 37.5 hour working weeks.  Even those who don’t regularly work evenings and weekends will almost certainly have flexible working patterns that make it even harder to estimate how long something will take.  This means that the standard units of staff time that are most often used – total days and percentage of full time – aren’t straightforwardly convertible from one currency to the other.  To make matters worse, some European funding sources seem to prefer ‘person months’.  Rather than the standard working week, I think this probably reflects the reality of academic work in many institutions:

The response to my question about how much time the project will take is often answered with another question.  What will the funder pay for?  What looks right?  What feels right?  Longer, thinner project, or shorter and more intensive?  The answer is always and inevitably, ‘it depends’.  It depends upon what is right for the project.  A longer less intensive project might make sense if you have to wait for survey responses to come in, or for other things to happen.  On the other hand, if it’s something that you can and want to work on intensively, go for it.  But the project has to come first.  What would it take to properly take on this research challenge?

Often the answer is a bit hand-wavy and guesstimate-ish.  Oh, I don’t know….say… about, two days per week?  Two and a half?  Would they fund three?  This is a generally a good starting point.  What’s not a good starting point is the kind of fifteen-minutes-every-Tuesday approach, where you end up with a ‘salami project’ – a few people involved, but sliced so thinly it’s hard to see how anything will get done.  This just isn’t credible, and it’s very unlikely to get funded by the likes of the ESRC (and other funders too, I’m sure) if they don’t believe that you’ll possibly be able to deliver for the amount of money (time) you’re asking for.  Either they’ll conclude that you don’t understand what your own research requires (which is fatal for any chance of funding), or they’ll think that you’re trying to pull a fast one in terms of the value for money criterion.  And they won’t like that much either.

The other way to make sure that you’re not going to get funded is to go the other way and become greedy.  I’ve noticed that – oddly – academics seldom over-estimate their own time, but over-estimate the amount of researcher time required.  I’ve seen potential bids before now that include a heavy smattering of research associates, but no clear idea what it actually is they’ll be doing all day, other than doing stuff that the lead investigator doesn’t want to do.  In a UK context, overheads are calculated on the basis of investigator and researcher time, so including researchers is doubly expensive.  One way round this that I often recommend is to ensure that what’s required is actually a research associate (who attracts overheads) rather than an academic-related project manager or administrator (who doesn’t).  But if it’s hard to estimate how long it will take you to do any given task, it’s doubly hard to estimate how long it will take someone else – usually someone less experienced and less skilled – and perhaps from a cognate sub-discipline rather than from your own.

My usual advice is for would-be principal investigators to draw up a table showing the various phases of the project as rows, and project staff as columns.  In addition to the main phases of the research, extra rows should be added for the various stages of dissemination and impact activity; for project coordination with colleagues; for line management of any researchers or administrative/managerial staff; for professional development where appropriate.  Don’t forget travelling time associated with meetings, fieldwork, conferences etc.  The main thing that I usually see underestimated is project management time.  As principal investigator, you will have to meet reasonably regularly with your finance people, and you’ll have to manage and direct the project research associates.  Far too many academics seem to see RAs as clones who will instinctively know what to do and don’t need much in the way of direction, advice, or feedback.

Once this is done, I suggest adding up the columns and then working backwards from those total numbers of project days to a percentage FTE, or days per week, or whatever alternative metric you prefer.  In the UK, the research councils assuming that 220 days = 1 working year, so you can use this to calculate the percentage of full time.  The number that you come out with should feel intuitively about right.  If it doesn’t, then something has probably gone wrong.  Go back to the table, and adjust things a little.  By going to and fro, from the precision of the table to the intuition of the percentage of time, you should reach what my philosophical hero and subject of my thesis, John Rawls, called ‘reflective equilibrium’.  Though he wasn’t talking about investigator time.  Once you’re happy with the result, you should probably use the figure from the table, and you should certainly consider putting the table in full in the ‘justification for resources’ section.

Something I’m starting at the moment as part of the end-of-project review process is to go back to the original estimates of staff time, and to get a sense from the research team about how accurate they were, and what they would estimate differently next time, if anything.  The two things that have come out strongly from this so far I’ve outlined above – managing staff and project administration – but I’ll be looking for others.

So…. over to you.  How do you estimate researcher and investigator time?  Have you been involved in a funded project?  If so, what did you miss in your forecasts (if anything)?  What would you do differently next time?

ESRC Demand Management Part 5: And the winner is…. researcher sanctions!

"And the prize for best supporting sanctions scheme goes to...."
And the winner is.....

The ESRC today revealed the outcome of the ‘Demand Management’ consultation, with the consultation exercise showing a strong preference for researcher sanctions rather than the other main options, which were institutional sanctions, institutional quotas, or charging for applications.  And therefore….

Given this clear message, it is likely that any further steps will reflect these views.

Which I think means that that’s what they’re going to do.  But being (a) academics, and (b) British, it has to be expressed in the passive voice and as tentatively as possible.

Individual researcher sanctions got the vote of  82% of institutional responses, 80% of learned society responses, and 44% of individual responses.   To put that in context, though, 32% of the individual responses were interpreted as backing none of the possible measures, which I don’t think was ever going to be a particular convincing response.    Institutional sanctions came second among institutions (11%), and institutional quotas (20%) among individual respondents.  Charging for applications was, as I expected, a non-starter, apparently attracting the support of two institutions and one learned society or ‘other agency’.  I’m surprised it got that many.

The issue of the presentation of the results as a ‘vote’ is an interesting one, as I don’t think that’s what this exercise was presented as at the time.  The institutional response that I was involved in was – I like to think – a bit more nuanced and thoughtful than just a ‘vote’ for one particular option.  In any case, if it was a vote, I’m sure that the ‘First Past the Post’ system which appears to have been used wouldn’t be appropriate – some kind of ‘alternative vote’ system to find the least unpopular option would surely have been more appropriate.  I’m also puzzled by the combining of the results from institutions, individuals, and learned societies into totals for ‘all respondents’ which seems to give the same weighting to individual and institutional responses.

Fortunately – or doubly-fortunately – those elements of the research community which responded delivered a clear signal about the preferred method of demand management, and, in my view at least, it’s the right one.  I’ll admit to being a bit surprised by how clear cut the verdict appears to be, but it’s very much one I welcome.

It’s not all good news, though.  The announcement is silent on exactly what form the programme of researcher sanctions will take, and there is still the possibility that sanctions may apply to co-investigators as well as the principal investigator.  As I’ve argued before, I think this would be a mistake, and would be grossly unfair in far too many cases.  I know that there are some non-Nottingham folks reading this blog, so if your institution isn’t one of the ones that responded (and remember only 44 of 115 universities did), it might be worth finding out why not, and making your views known on this issue.

One interesting point that is stressed in the announcement is that individual researcher sanctions – or any form of further ‘demand management’ measures – may never happen.  The ESRC have been clear about this all along – the social science research community was put on notice about the unsustainablity of the current volume of applications being submitted, and that a review would take place in autumn 2012.  The consultation was about the general form of any further steps should they prove necessary.  And interestingly the ESRC are apparently ‘confident’ they they will not.

We remain confident that by working in partnership with HEIs there will be no need to take further steps. There has been a very positive response from institutions to our call for greater self-regulation, and we expect that this will lead to a reduction in uncompetitive proposals.

Contrast that with this, from March, when the consultation was launched:

We very much hope that we will not need additional measures.

Might none of this happen?  I’d like to think not, but I don’t share their confidence, and I fear that “very much hope” was nearer the mark.  I can well believe that each institution is keen to up its game, and I’m sure discussions are going on about new forms of internal peer review, mentoring, research leadership etc in institutions all across the country.  Whether this will lead to a sufficient fall in the number of uncompetitive applications, well, I’m not so sure.

I think there needs to be an acceptance that there are plenty of perfectly good research ideas that would lead to high quality research outputs in quality journals, perhaps with strong non-academic impact, which nevertheless aren’t ‘ESRC-able’ – because they’re merely ‘very good’ or ‘excellent’ rather than ‘outstanding’.  And it’s only the really outstanding ideas that are going to be competitive.  If all institutions realise this, researcher sanctions may never happen.  But if hubris wins out, and everyone concludes that it’s everyone else’s applications that are the problem, then researcher sanctions are inevitable.

A visit from the British Academy….

The British Academy logo, featuring the Greek Muse Clio, according to wikipedia...
The British Academy

Ken Emond, Head of Research Awards of the British Academy, came to visit the University of Nottingham the other week to talk about the various and nefarious research funding schemes that are on offer from the British Academy.  To make an event of it, my colleagues in the Centre for Advanced Studies also arranged for various internal beneficiaries of the Academy’s largesse to come and talk about the role that Academy funding had had in their research career.  I hope no-one minds if I repeat some of the things that were said – there was no mention of ‘Chatham House’ rules or of ‘confidential learning agreements’, and I don’t imagine that Ken gives privileged information to the University of Nottingham alone, no matter how wonderful we are.

Much of what funders’ representatives tend to say during institutional visits or AMRA conferences is pretty much identical to the information already available on their website in one form or another, but it’s interesting how many academics seem to prefer to hear the information in person rather than read it in their own time.  And it’s good to put a face to names, and faces to institutions.  Although I think I shall probably always share Phil Ward‘s mental image of the BA as an exclusive Rowley Birkin QC-style private members club.  But it’s good to have a reminder of what’s on offer, and have an opportunity to ask questions.

I met Ken very briefly at the ARMA conference in 2010, and his enthusiasm for the Small Grants Scheme then (and now) was obvious.  I was very surprised when it was scrapped, and it seems likely that this was imposed rather than freely chosen.  However, it’s great to see it back again, and this time including support for conference funding to disseminate the project findings.  It seems the call is going to be at least annual, with no decision taken yet on whether there will be a second call this year, as in previous years.

It seems much more sensible than having separate schemes for projects and for conference funding.  It’s unlikely that we’re going to see a return of the BA Overseas Conference Scheme, but…. it was quite a lot of work in writing and assessing for really very small amounts of money.  Although having said that, when I was at Keele those very small amounts of money really did help us send researchers to prestigious conferences (especially in the States) they wouldn’t otherwise have attended.

One of the questions asked was about the British Academy’s attitude to demand management, of the kind that the EPSRC have introduced and that the ESRC are proposing.  The response was that they currently have no plans in this direction – they don’t think that any institutions are submitting an excessive number of applications.

Although the British Academy has some of the lowest success rates in town for its major schemes, they are all light touch applications – certainly compared to the Research Councils.  Mid-Career and Post-Doc Fellowships both have an outline stage, and the Senior Research Fellowships application form is hardly more taxing than a Small Grant one.  Presumably they’re also quick and easy to review – I wonder how many of those a referee could get through in the time it took them to review a single Research Council application?  Which does raise the suggestion from Mavan, a commenter on one of my previous posts, about cutting the ESRC application form dramatically.

But… it’s possible that the relative brevity of the application forms is itself increasing the number of applications, and that’s certainly something that the ESRC were concerned about when considering their own move to outline stage applications.

I guess a funding scheme could be credible and sustainable with a low success rate and a low ‘overhead’ cost of writing and reviewing applications or a high success rate with a high overhead cost.  The problem is when were get to where we are at the moment with the ESRC, with low success rates and high overhead costs.

Yet another ‘oh look, the start of term’ blog post….

Apologies for the lack of posts recently.  I’ve been off on leave for a couple of weeks, but although this blog is written in my own time and in a personal capacity, I decided to ended up taking a complete break from all things research funding related.  And yes, I did have a nice break, thanks for asking…. part ‘stay-cation’ and part ‘prepare for house move that won’t now take place this leave year after all’


“Hello! Hello! It’s good to be back!”

I managed to miss the first week of term, although the return of the students is fairly hard to miss in university cities like Nottingham.  Suddenly there are young people everywhere, and about a third of them look lost.  I played my part in supporting the student induction experience by giving directions to an undergraduate who had lost herself between two of the University of Nottingham campuses (campi?).  Easily done.  This is usually been the limit of my interaction with undergraduates, other than telling them that, no, I don’t know the code to the computer room, and that they should ask at reception.

Universities are strange, almost depressing, places outside term time.  A little bit like I’d imagine the whole world would be after a ‘rapture’ of the kind that some odd kinds of Christians are expecting.  Sure, it’s nice for a day or so to have the place to ourselves, but when the students go, so does the infrastructure.  Limited choice of sandwiches at lunchtime, a reduced bus service, and of course, the staff slope off as well.  Academics for a combination of annual leave and research time (except this year, of course.  Thanks, ESRC, for those September deadlines.   Thank you so much), and the rest of us will also look to take the bulk of our leave then too.  On one level, you’d think it would a good time to get things done, but on the other, the people you need to get on board to get any of them done tend not to be around.  And as we’ve seen, no time of the year is really any good.

Does anyone else play the ‘out of office’ lottery?  Trying to predict how many out of office emails you’ll get in a day, or in response to any one particular email.  (On the subject of which, wouldn’t it be handy to have an ‘oh, never mind, enjoy yourself’ option to respond to o-o-o emails with, so that you could delete your original email so they’d have one less to deal with when they return.  I’d also quite like an “I’ve told you once already” o-o-o-email which subtly escalates in annoyance if more emails are received from the same person).

But it’s remarkable how soon the spring in the step fades, even on a warm October morning.  The campus is bustling with activity, when academic and non-academic colleagues are around (if busy) and the corridors are full of students’ chatter.  Office doors everywhere are left just a little bit ajar, colleagues are catching up on their summer holidays research, buses are more frequent (if a little less reliable), optimism and excitement are in the air.and the university feels, well, like a university again.

But then I’m asked for the code to the computer room before I even get as far as unlocking my office door, I have to queue for ten minutes for a sandwich at lunchtime, I can’t get on to the hopper bus after a meeting and have to walk back to base, and corridors are blocked with lost or dawdling students, or just ‘hanging out’.  Though I suspect no-one says ‘hangs out’ any more.  And then I start to yearn for the peace and quiet of the summer.  Almost.

Not really…. Hello, Nottingham University Business School.  Hello, term time.  It’s good to be back.

University Life: Why now is the wrong time of the academic year to get anything done….

A picture of a calendar
Better luck next year?

… and why that’s true for absolutely any value of “now”…..

September
“It’s the start of the academic year soon, everyone’s concentrating on preparing their teaching”

October
“It’s the busiest time for teaching.  I’ve got 237 tutorials this week alone”

November
“I’ve got about 4,238 essays to mark.”

December
“It’s nearly Christmas, nothing gets done at this time of year”

January
“I’ve got sixty thousand exam scripts to mark”

February
[See October]

March-April
“With the Easter break coming up, well…”

May
[See January, but with added Finalist-related urgency and some conferences]

June-August
“Conference season…. annual leave…. concentrated period of research in Tuscany