Yes, I know I used the same picture last year. You can write to the usual address for your money back….
Hello everyone, and happy new year’s eve. Or probably more likely by the time you’re reading this, happy first day back at work of 2013 and a prosperous new email backlog from people who had less time off over Christmas than you, and are anxious to demonstrate their productivity. My last new year’s message was a bit of a whingeathon, so I’m going to be more positive this season and share some youtubes that I’ve enjoyed over the last year. I know you’ve got a lot to do today, but why not leave this page open and watch the clips over lunch?
1. John Cleese, Jonathan Miller – Words… and things
This is a sketch from 1977 starring John Cleese and Jonathan Miller, which I think I’ve tweeted before with the title “Philosophers preparing their REF Impact statement”. And while there’s a bit of that, what I like most about this is the superbly well observed and subtly exaggerated academic mannerisms. Are those mannerisms a peculiar philosophy affectation, or are they more widespread?
2. Armstrong and Miller Physics Special
In which Ben Miller (who I think has a PhD in physics) demonstrates how not to do public engagement/media work. Watching this, it’s hard not to appreciate the effort that does go into communicating very complex science to the general public, particularly the efforts to explain the search for the Higgs via the medium of rap, when I suspect that the reality is pretty much as Miller’s character says. Special hat tip on the science public engagement front to m’colleagues from the Periodic Videos team at the University of Nottingham’s School of Chemistry, though apparently they prefer Dubstep (whatever that is) to rap music.
3. A Very Peculiar Practice
I finally got round to watching this late 1980s TV series about a medical practice at a university. It’s both very current (debates about research v. teaching; working with industry; student finances; university politics; the role of the university; the place of the arts/humanities) and very dated (haircuts; weird theme music and opening credits; accents – some weird London accents that have either died out or never existed at all). On the down side, it does require the viewer to accept the premise that a university medical practice is a department of the university (was this ever the case anywhere?), and the overall tone and level of (sur)realism uneasily shifts between sitcom and comedy-drama. On the up side, it’s an interesting view of 1980s campuses (Birmingham and Keele – my former stomping ground) and has a superb cast – Peter Davison, Barbara Flynn, John Bird, plus small early roles for Hugh Grant and Kathy Burke. It’s worth a look – I’ve embedded the trailer for the DVD complete box set, though a fair bit of it is also on youtube if you want more of a taster before investing.
4. “Don’t wanna work in admin”
I’ve been a fan of Nick Helm’s brand of on-the-edge-of-a-breakdown stand-up and musical comedy since seeing him in Nottingham a few years back – hilarious and terrifying at the same time. What I remember most about that performance was a song that will resonate with anyone who has or has had a basic admin job. It’s very sweary and therefore not work safe, so I’m only going to link it rather than embed it.
Update – read this post for the 2012/13 stats for success rates by discipline
The ESRC have recently published a set of ‘vital statistics‘ which are “a detailed breakdown of research funding for the 2011/12 financial year” (see page 22). While differences in success rates between academic disciplines are nothing new, this year’s figures show some really quite dramatic disparities which – in my view at least – require an explanation and action.
The overall success rate was 14% (779 applications, 108 funded) for the last tranche of responsive mode Small Grants and response mode Standard Grants (now Research Grants). However, Business and Management researchers submitted 68 applications, of which 1 was funded. One. One single funded application. In the whole year. For the whole discipline. Education fared little better with 2 successes out of 62.
Just pause for a moment to let that sink in. Business and Management. 1 of 68. Education. 2 of 62.
Others did worse still. Nothing for Demographics (4 applications), Environmental Planning (8), Science and Technology Studies (4), Social Stats, Computing, Methods (11), and Social Work (10). However, with a 14% success rate working out at about 1 in 7, low volumes of applications may explain this. It’s rather harder to explain a total of 3 applications funded from 130.
Next least successful were ‘no lead discipline’ (4 of 43) and Human Geography (3 from 32). No other subjects had success rates in single figures. At the top end were Socio-Legal Studies (a stonking 39%, 7 of 18), and Social Anthropology (28%, 5 from 18), with Linguistics; Economics; and Economic and Social History also having hit rates over 20%. Special mention for Psychology (185 applications, 30 funded, 16% success rate) which scored the highest number of projects – almost as many as Sociology and Economics (the second and third most funded) combined.
Is this year unusual, or is there a worrying and peculiar trend developing? Well, you can judge for yourself from this table on page 49 of last year’s annual report, which has success rates going back to the heady days of 06/07. Three caveats, though, before you go haring off to see your own discipline’s stats. One is that the reports refer to financial years, not academic years, which may (but probably doesn’t) make a difference. The second is that the figures refer to Small and Standard Grants only (not Future Leaders/First Grants, Seminar Series, or specific targeted calls). The third is that funded projects are categorised by lead discipline only, so the figures may not tell the full story as regards involvement in interdisciplinary research.
You can pick out your own highlights, but it looks to me as if this year is only a more extreme version of trends that have been going on for a while. Last year’s Education success rate? 5%. The years before? 8% and 14% Business and Management? A heady 11%, compared to 10% and 7% for the preceding years. And you’ve got to go all the back to 9/10 to find the last time any projects were funded in Demography, Environmental Planning, or Social Work. And Psychology has always been the most funded, and always got about twice as many projects as the second and third subjects, albeit from a proportionately large number of applications.
When I have more time I’ll try to pull all the figures together in a single spreadsheet, but at first glance many of the trends seem similar.
So what’s going on here? Well, there are a number of possibilities. One is that our Socio Legal Studies research in this country is tip top, and B&M research and Education research is comparatively very weak. Certainly I’ve heard it said that B&M research tends to suffer from poor research methodologies. Another possibility is that some academic disciplines are very collegiate and supportive in nature, and scratch each other’s backs when it comes to funding, while other disciplines are more back-stabby than back-scratchy.
But are any or all of these possibilities sufficient to explain the difference in funding rates? I really don’t think so. So what’s going on? Unconscious bias? Snobbery? Institutional bias? Politics? Hidden agendas? All of the above? Anyone know?
More pertinently, what do we do about it? Personally, I’d like to see the appropriate disciplinary bodies putting a bit of pressure on the ESRC for some answers, some assurances, and the production of some kind of plan for addressing the imbalance. While no-one would expect to see equal success rates for every subject, this year’s figures – in my view – are very troubling.
And something needs to be done about it, whether that’s a re-thinking of priorities, putting the knives away, addressing real disciplinary weaknesses where they exist, ring-fenced funding, or some combination of all of the above. Over to greater minds than mine…..
The Association of Research Managers and Administrators conference was held in Southampton last week, and I’ve only got time to scribble a few words about it. It’s a little frustrating, really – I’ve come back from the conference with various ideas and schemes for work, and a few for the blog, but I’m on annual leave until the end of July. While I’ve always written this blog in my own time, I’m going to have a near-complete break (apart from perhaps a little Twitter lurking) so my reader will have to wait until July at the very earliest for the second instalment of my impact series.
I have a slight stammer that I’m told that most people don’t notice, so I’m not a ‘natural’ public speaker, but I’m very pleased with the way that the session went. I’m very grateful to my three co-presenters for their efforts and for what really amounted to quite a lot of preparation time, including a meeting in London. I’m also very grateful to the delegates who attended – I think I counted 50 or so, which for the final session of the conference and scheduled against a very strong line-up of parallel sessions, was pretty good. It was a very warm afternoon, but energy and attention levels in the room felt high, and this helped enormously. So if you made it, thank you for coming, thank you for your attention, and most importantly of all, thank you for laughing at our jokes.
David opened the session by asking about the audience’s experience with social media, I was surprised at how much experience there was in the room. We weren’t far short of 100% on Facebook, probably about 20% or more on or having using Twitter, and four or five bloggers. Perhaps it shouldn’t have been a surprise, as perhaps the title of the session would have particularly appealed to those with an interest or previous experience. But it was good to have an idea of the level to pitch things.
The session consisted of a brief introduction and explanation of social media, followed by four case studies. Phil and I talked about our motivations in setting up our own blogs, our experiences, lessons learnt, and benefits and challenges. Julie and David talked about their experience in setting up institutional research blogs, and how they went about getting institutional acceptance and academic buy-in. It was interesting to see that the Open University had a poster presentation about a research blog that they’ve set up, though that’s internal only at the moment. ARMA itself is now on Twitter, and this was the first year that the conference had an official hashtag – #ARMA2012. While there’s no need for an official one – sometimes they just emerge – it’s very helpful to have an element of coordination. I don’t think blogging or social media are going away any time soon, and I can only see their usage increasing. – though I do have reservations about scalability and sustainability.
As I said in the presentation, my motivations in setting up a blog were to try to join in a broader conversation with academics, funders, and people like me. We get to do a lot of that at the annual ARMA conference, but it would be good to keep that going throughout the rest of the year too. A secondary motivation was to learn by doing – I’m expected to help academics write their pathways to impact, which almost inevitably involve social media, and by getting involved myself I understand it in a way that I could never have understood as a mere bystander.
My blog is now a few weeks shy of its first birthday, an auspicious event marked by a birthday card invoice from my hosting provider, and a time for reflection. I’ve managed reasonably well to hit an average of 2-3 posts per month – some reactions to news, some more detailed think pieces, and some lighter reflections on university culture and life. That’s not too bad, but looking into the future I wonder whether I’ll be able to sustain this, and whether I’ll want to spend my own time writing about these things. While I’m hopeful that I might be able to shift a little of the blog into my ‘day job’ (discussions on that to follow), one other option is to share the load, and I think the future for most blogs is multi-author. Producing semi-regular, consistent quality content is a challenge, and I’m going to be soliciting guest posts in the future to feature alongside my own – whether that’s semi-regular or one off. So, if you’d like to write occasionally but don’t want a whole blog, this might be a good opportunity. Happy to discuss anything that’s a good fit with the overall theme of the blog. Please drop me an email if you’re interested – I don’t bite.
One issue that came up in the questions (and afterwards on Twitter), was the question of the personal and the professional. My sense was that a fair few people in the room had their own Twitter accounts already, but used them for personal purposes, rather than for professional purposes, and were concerned about mixing the two. Probably there was little or no reference to their job in their bio, and they tweet about their interests and talk to family and friends. This issue of the personal and the professional was something we touched on only very briefly in our talk, and mainly in reference to blogs rather than Twitter. But it’s clearly something that concerns people, and may be an active barrier to more people getting involved in Twitter conversations. Probably the one thing I’d do differently about the presentation would be to say more about this, and I’ve added it to my list of topics for blog posts for the future.
If your research leads directly or indirectly to this, we'll be having words.....
Partly inspired by a twitter conversation and partly to try to bring some semblance of order my own thoughts, I’m going to have a go about writing about impact. Roughly, I’d argue that:
The impact agenda is – broadly – a good thing
Although there are areas of uncertainty and plenty of scope for collective learning, I think the whole area is much less opaque than many commentators seem to think
While the Research Councils and the REF have a common definition of ‘impact’, they’re looking at it from different ends of the telescope.
This post will come in three parts. In part one, I’ll try to sketch a bit of background and say something position of impact in the REF. In part two, I’ll turn to the Research Councils and think about how ‘impact’ differs from previous different – but related – agendas. In part three, I’ll pose some questions that are puzzling me about impact and test my thinking with examples.
Why Impact?
What’s going on? Where’s it come from? What’s driving it? I’d argue that to understand the impact agenda properly, it’s important to first understand the motivations. Broadly speaking, I think there are two.
Firstly, I think it arises from a worry about a gap between academic research and those who might find it useful in some way. How may valuable insights of various kinds from various disciplines have never got further than an academic journal or conference? While some academics have always considered providing policy advice or writing for practitioner journals as a key part of their role as academics, I’m sure that’s not universally true. I can imagine some of these researchers now complaining like music obsessives that they were into impact before anyone else and it sold out and went all mainstream. As I’ve argued previously, one advantage of the impact agenda is that it gives engaged academics some long overdue recognition, as well as a much greater incentive for others to become involved in impact related activities.
Secondly, I think it’s about finding concrete, credible, and communicable evidence of the importance and value of academic research. If we want to keep research funding at current levels, there’s a need to show return on investment and that the taxpayer is getting value for money. Some will cringe at the reduction of the importance and value of research to such crude and instrumentalist terms, but we live in a crude and instrumentalist age. There is an overwhelming case for the social and economic benefits of research, and that case must be made. Whether we like it or not, no government of any likely hue is just going to keep signing the cheques. The champions of research in policy circles do not intend to go naked into the conference chamber when they fight our corner. To what extent the impact agenda comes directly from government, or whether it’s a pre-emptive move, I’m not quite sure. But the effect is pretty much the same.
What’s Impact in the REF?
The REF definition of impact is as follows:
140. For the purposes of the REF, impact is defined as an effect on, change or benefit to the economy, society, culture, public policy or services, health, the environment or quality of life, beyond academia (as set out in paragraph 143).
141. Impact includes, but is not limited to, an effect on, change or benefit to:
• the activity, attitude, awareness, behaviour, capacity, opportunity, performance, policy, practice, process or understanding
• of an audience, beneficiary, community, constituency, organisation or individuals
• in any geographic location whether locally, regionally, nationally or internationally.
142. Impact includes the reduction or prevention of harm, risk, cost or other negative effects.
Assessment Framework and Guidance on Submissions, page 26.
Paragraph 143 goes on to rule out academic impact on the grounds that it’s assessed in the outputs and environment section. Fair enough. More controversially, it goes on to state that “impacts on students, teaching, and other activities within the submitting HEI are excluded”. But it’s possible to understand the reasoning. If it were included, there’s a danger that far too impact case studies would be about how research affects teaching – and while that’s important, I don’t think we’d want it to dominate. There’s also an argument that the link between research and teaching ought to be so obvious that there’s no need to measure it for particular reward. In practical terms, I think it would be hard to measure. I might know how my new theory has changed how I teach my module on (say) organisational behaviour to undergraduates, but it would be hard to track that change across all UK business schools. I’d also worry about the possible perverse incentives on the shape of the curriculum that allowing impact on teaching might create.
The Main Panel C (the panel for most social sciences) criteria state that:
The main panel acknowledges that impact within its remit may take many forms and occur in a wide range of spheres. These may include (but are not restricted to): creativity, culture
and society; the economy, commerce or organisations; the environment; health and welfare; practitioners and professional services; public policy, law and services.
The categories used to define spheres of impact, for the purpose of this document, inevitably overlap and should not be taken as restrictive. Case studies may describe impacts which have affected more than one sphere. (para 77, pg. 68)
There’s actually a lot of detail and some good illustrations of what forms impact might take, and I’d recommend having a read. I wonder how many academics not directly involved in REF preparations have read this? One difficulty is finding it – it’s not the easiest document to track down. For my non-social science reader(s), the other panel working methods can be found here. Helpfully, nothing on that page will tell you which panel is which, but (roughly) Panel A is health and life sciences; B is natural sciences, computers, maths and engineering; C is social science; and D humanities. Each panel criteria document has a table with examples of impact.
What else do we know about the place of impact in the REF? Well, we know that impact has to have occurred in the REF period (1 January 2008 to 31 July 2013) and that impact has to be underpinned by excellent research (at least 2*) produced at the submitting university at some point between 1 January 1993 and 31 December 2013. It doesn’t matter if the researchers producing the research are still at the institution – while publications move with the author, impact stays with the institution. However, I can’t help wondering if an excessive reliance on research undertaken by departed staff won’t look too much like trading on past glories. But probably it’s about getting the balance right. The number of case studies required is approximately 1 per 8 FTE submitted, but see page 28 of the guidance document for a table.
Impact will have a weighting of 20%, with environment 15% and outputs (publications) 65%, and it looks likely that the weighting of impact will increase next time. However, I wouldn’t be at all surprised if the actual contribution ends up being less than that. If there’s a general trend that overall scores for impact are lower than that of (say) publications, then the contribution will end up being less than 20%. My understanding is that for some units of assessment, environment was consistently rated more highly, thus de facto increasing the weighting. Unfortunately this is just a recollection of something I read years ago, and which I can’t now find. But if this is right, and if impact does come in with lower marks overall, we neglect environment at our peril.
Interesting article and leader in this week’s Times Higher on the topic of impact, both of which carry arguments that “university managers” have over-reacted to the impact agenda. I’m not sure whether that’s true or not, but I suspect that it’s all a bit more complicated than either article makes it appear.
The article quotes James Ladyman, Professor of Philosophy at the University of Bristol, as saying that university managers had overreacted and created “an incentive structure and environment in which an ordinary academic who works on a relatively obscure area of research feels that what they are doing isn’t valued”.
If that’s happened anywhere, then obviously things have gone wrong. However, I do think that this need to be understood in the context of other groups and sub-groups of academics who likewise feel – or have felt – undervalued. I can well understand why academics whose research does not lend itself to impact activities would feel alienated and threatened by the impact agenda, especially if it is wrongly presented (or perceived) as a compulsory activity for everyone – regardless of their area of research, skills, and comfort zone – and (wrongly) as a prerequisite for funding.
Another group of researchers who felt – and perhaps still feel – under-valued are those undertaking very applied research. It’s very hard for them to get their stuff into highly rated (aka valued) journals. Historically the RAE has not been kind to them. The university promotions criteria perhaps failed to sufficiently recognise public engagement and impact activity – and perhaps still does. While all the plaudits go to their highly theoretical colleagues, the applied researchers feel looked down upon, and struggle to get academic recognition. If we were to ask academics whose roles are mainly teaching (or teaching and admin) rather than research, I think we may find that they feel undervalued by a system which many of them feel is obsessed by research and sets little store on excellent (rather than merely adequate) teaching. Doubtless increased fees will change this, and perhaps we will hear complaints of the subsequent under-valuing of research relative to teaching.
So if academics working in non-impact friendly (NIFs, from now on) areas of research are now feeling under-valued, they’re very far from alone. It’s true that the impact agenda has brought about changes to how we do things, but I think it could be argued that it’s not that the NIFs are now under valued, but that other kinds of research and academic endeavour – namely applied research and impact activities (ARIA from now on) – are now being valued to a greater degree than before. Dare I say it, to an appropriate degree? Problem is, ‘value’ and ‘valuing’ tends to be seen as a zero sum game – if I decide to place greater emphasis on apples, the oranges may feel that they have lost fruit bowl status and are no longer the, er, top banana. Even if I love oranges just as much as before.
Exactly how institutions ‘value’ (whatever we mean by that) NIF research and ARIA is an interesting question. It seems clear to me that an institution/school/manager/grant giving body/REF/whatever could err either way by undervaluing and under-rewarding either. We need both. And we need excellent teachers. And – dare I say it – non-academic staff too. Perhaps the challenge for institutions is getting the balance right and making everyone feel valued, and reflecting different academic activities fairly in recruitment and selection processes and promotion criteria. Not easy, when any increased emphasis on any one area seem to cause others to feel threatened.
Update – read 
The 
Interesting