The consequences of Open Access: Part 1: Is anyone thinking about the “lay” reader?

The thorny issue of “open access” – which I take to mean the question of how to make the fruits of publicly-funded research freely and openly available to the public – is one that’s way above my pay grade and therefore not one I’ll be resolving in this blog post.  Sorry about that.  I’ve been following the debates with some interest, though not, I confess, an interest which I’d call “keen” or “close”.  No doubt some of the nuances and arguments have escaped me, and so I’ll be going to an internal event in a week or so to catch up.  I expect it’ll be similar to this one helpfully written up by Phil Ward over at Fundermentals.  Probably the best single overview of the history and arguments about open access is an article in this week’s Times Higher article by Paul Jump – well worth a read.

I’ve been wondering about some of the consequences of open access that I haven’t seen discussed anywhere yet.  This first post is about the needs of research users, and I’ll be following it up with a post about what some consequences of open access for academics that may require more thought.

I wonder if enough consideration is being given to the needs and interests of potential readers and users of all this research which is to be liberated from paywalls and other restrictions.  It seems to me that if Joe Public and Joanna Interested-Professional are going to be able to get their mitts on all this research, then this has very serious implications for academic research and academic writing.  I’d go as far as to say it’s potentially revolutionary, and may require radical and permanent changes to the culture and practice of academic writing for publication in a number of research fields.  I’m writing this to try to find out what thought has been given to this, amidst all the sound and fury about green and gold.

If I were reading an academic paper in a field that I was unfamiliar with, I think there are two things I’d struggle with.  One would be properly and fully understanding the article in itself, and the second would be understanding the article in the context of the broader literature and the state of knowledge in that area.  By way of example, a few years back I was looking into buying a rebounder – a kind of indoor mini-trampoline.  Many vendors made much of a study attributed to NASA which they interpreted as making dramatic claims about the efficacy of rebounder exercising compared to other kinds of exercise.  Being of a sceptical nature and armed with campus access to academic papers that weren’t open access, I went and had a look myself.  At the time, I concluded that these claims weren’t borne out by the study, which was really aimed at looking at helping astronauts recover from spending time in weightlessness.  I don’t have access to the article as I’m writing this, so I can’t re-check, but here’s the abstract.  I see that this paper is over 30 years old, and that eight people is a very small sample size…. so… perhaps superseded and not very highly powered.  I think the final line of the abstract may back up my recollection (“… a finding that might help identify acceleration parameters needed for the design of remedial procedures to avert deconditioning in persons exposed to weightlessness”).

For the avoidance of doubt, I infer no dishonesty nor nefarious intent on the part of rebounder vendors and advocates – I may be wrong in my interpretation, and even if I’m not, I expect this is more likely to be a case of misunderstanding a fairly opaque paper rather than deliberate distortion.   In any case, my own experience with rebounders has been very positive, though I still don’t think they’re a miracle or magic bullet exercise.

How would open access help me here?  Well, obviously it would give me access to the paper.  But it won’t help me understand it, won’t help me draw inferences from it, won’t help me place it in the context of the broader literature.  Those numbers in that abstract look great, but I don’t have the first clue what they mean.  Now granted, with full open access I can carry out my own literature search if I have the time, knowledge and inclination.  But it’ll still be difficult for me to compare and contrast and form my own conclusions.  And I imagine that it’ll be harder still for others without a university education and a degree of familiarity with academic papers, or who haven’t read Ben Goldacre’s excellent Bad Science.

I worry that open access will only make it easier for people with an agenda (to sell products, or to push a certain political agenda) to cherry-pick evidence and put together a new ill-deserved veneer of respectability by linking to academic papers and presenting (or feigning to present) a summary of their contents and arguments.  The intellectually dishonest are already doing this, and open access might make it easier.

I don’t present this as an argument against open access, and I don’t agree with a paternalist elitist view that holds that only those with sufficient letters after their name can be trusted to look at the precious research.  Open access will make it easier to debunk the charlatans and the quacks, and that’s a good thing.  But perhaps we need to think about how academics write papers from now on – they’re not writing just for each other and for their students, but for ordinary members of the public and/or research users of various kinds who might find (or be referred to) their paper online.  Do we need to start thinking about a “lay summary” for each paper to go alongside the abstract, setting out what the conclusions are in clear terms, what it means, and what it doesn’t mean?

What do we do with papers that present evidence for a conclusion that further research demonstrates to be false?  In cases of research misconduct, these can be formally withdrawn, but we wouldn’t want to do that in cases of papers that have just been superseded, not least because they might turn out to be correct after all, and are still a valid and important part of the debate.  Of course, where the current scientific consensus on any particular issue may not be clear, and it’s less clear still how the state of the debate can be impartially communicated to research users.

I’d argue that we need to think about a format or template for an “information for non-academic readers” or something similar.  This would set out a lay summary of the research, its limitations, links to key previous studies, details of the publishing journal and evidence of its bona fides.  Of course, it’s possible that what would be more useful would be regularly written and re-written evidence briefings on particular topics designed for research users.  One source of lay reviews I particularly like is the NHS Behind the Headlines which comments on the accuracy (or otherwise) of media coverage of health research news.  It’s nicely written, easily accessible, and isn’t afraid to criticise or praise media coverage when warranted.  But even so, as the journals are the original source, some kind of standard boiler plate information section might be in order.

Has there been any discussion of these issues that I’ve missed?  This all seems important to me, and I wouldn’t want us to be in a position of finally agreeing what colour our open access ought to be, only to find that next to no thought has been given to potential readers.  I’ve talked mainly about health/exercise examples in this entry, but all this could apply  just as well to pretty much any other field of research where non-academics might take an interest.

ESRC success rates by discipline: what on earth is going on?

Update – read this post for the 2012/13 stats for success rates by discipline

The ESRC have recently published a set of ‘vital statistics‘ which are “a detailed breakdown of research funding for the 2011/12 financial year” (see page 22).  While differences in success rates between academic disciplines are nothing new, this year’s figures show some really quite dramatic disparities which – in my view at least – require an explanation and action.

The overall success rate was 14% (779 applications, 108 funded) for the last tranche of responsive mode Small Grants and response mode Standard Grants (now Research Grants).  However, Business and Management researchers submitted 68 applications, of which 1 was funded.  One.  One single funded application.  In the whole year.  For the whole discipline.  Education fared little better with 2 successes out of 62.

Just pause for a moment to let that sink in.  Business and Management.  1 of 68.  Education.  2 of 62.

Others did worse still.  Nothing for Demographics (4 applications), Environmental Planning (8), Science and Technology Studies (4), Social Stats, Computing, Methods (11), and Social Work (10).  However, with a 14% success rate working out at about 1 in 7, low volumes of applications may explain this.  It’s rather harder to explain a total of 3 applications funded from 130.

Next least successful were ‘no lead discipline’ (4 of 43) and Human Geography (3 from 32).  No other subjects had success rates in single figures.  At the top end were Socio-Legal Studies (a stonking 39%, 7 of 18), and Social Anthropology (28%, 5 from 18), with Linguistics; Economics; and Economic and Social History also having hit rates over 20%.  Special mention for Psychology (185 applications, 30 funded, 16% success rate) which scored the highest number of projects – almost as many as Sociology and Economics (the second and third most funded) combined.

Is this year unusual, or is there a worrying and peculiar trend developing?  Well, you can judge for yourself from this table on page 49 of last year’s annual report, which has success rates going back to the heady days of 06/07.  Three caveats, though, before you go haring off to see your own discipline’s stats.  One is that the reports refer to financial years, not academic years, which may (but probably doesn’t) make a difference.  The second is that the figures refer to Small and Standard Grants only (not Future Leaders/First Grants, Seminar Series, or specific targeted calls).  The third is that funded projects are categorised by lead discipline only, so the figures may not tell the full story as regards involvement in interdisciplinary research.

You can pick out your own highlights, but it looks to me as if this year is only a more extreme version of trends that have been going on for a while.  Last year’s Education success rate?  5%.  The years before?  8% and 14%  Business and Management?  A heady 11%, compared to 10% and 7% for the preceding years. And you’ve got to go all the back to 9/10 to find the last time any projects were funded in Demography, Environmental Planning, or Social Work.  And Psychology has always been the most funded, and always got about twice as many projects as the second and third subjects, albeit from a proportionately large number of applications.

When I have more time I’ll try to pull all the figures together in a single spreadsheet, but at first glance many of the trends seem similar.

So what’s going on here?  Well, there are a number of possibilities.  One is that our Socio Legal Studies research in this country is tip top, and B&M research and Education research is comparatively very weak.  Certainly I’ve heard it said that B&M research tends to suffer from poor research methodologies.  Another possibility is that some academic disciplines are very collegiate and supportive in nature, and scratch each other’s backs when it comes to funding, while other disciplines are more back-stabby than back-scratchy.

But are any or all of these possibilities sufficient to explain the difference in funding rates?  I really don’t think so.  So what’s going on?  Unconscious bias?  Snobbery?  Institutional bias?  Politics?  Hidden agendas?  All of the above?  Anyone know?

More pertinently, what do we do about it?  Personally, I’d like to see the appropriate disciplinary bodies putting a bit of pressure on the ESRC for some answers, some assurances, and the production of some kind of plan for addressing the imbalance.  While no-one would expect to see equal success rates for every subject, this year’s figures – in my view – are very troubling.

And something needs to be done about it, whether that’s a re-thinking of priorities, putting the knives away, addressing real disciplinary weaknesses where they exist, ring-fenced funding, or some combination of all of the above.  Over to greater minds than mine…..

On strike…..

"Careful Now"
"Down with this sort of thing!"

I hate having to take strike action.  I hate having to take action short of a strike, which recently involved the highly radical step of, er, working to contract.

I particularly hate it at the moment because tomorrow will be my second anniversary at Nottingham University Business School.  I think I’m very lucky to be at a well-run university and in a well-run School.  I admire and respect my colleagues, and have no reason to think that that respect isn’t returned.  I enjoy my work – challenging enough to stretch me, not so stressful that it might break me.  I hope these words won’t come back to haunt me, but for now, I consider myself to be very, very lucky.

So I don’t want to strike.  I also don’t want to ‘politicise’ my blog by saying too much about it.  Not least because it’s hard to get to the bottom of what’s really going on.  I’ve foolishly  neglected to become an expert in pensions, and so I don’t fully grasp the issues.  I know enough not to take at face value the information that the employers are giving us, nor the information from the UCU.  On the one hand, it’s hard not to conclude that (regardless of your personal politics) that the government is doing all kinds of things that it’s secretly wanted to do for ages under the guise of TINA (‘There Is No Alternative’).  It’s also hard to avoid the fact that changes were made to our pension scheme not so long ago that were supposed to address the (undoubted) issues of longer life expectancy.  So it’s hard not to wonder why we’re back again so soon.  And hard not to wonder how long it will be before we’re back revisiting and adjusting again.  And again.  And again.

There’s something of the theatrical about all of the public posturing and negotiations and the wars of words and the spin that goes on with every industrial dispute.  Often I think what’s really going on is not what it seems.  “Offers” are made which are intended to be rejected, and in the full knowledge that a better offer will be made after the inevitable industrial action.  Unions ask for more than they could possibly expect to get.  In the end, we usually end up with an agreement which lets both sides claim victory and appease their constituency.  But what we will have tomorrow is a show of the strength of feeling and stomach for a fight.  It may or not make any difference in the short term.  But in the long term, it sends a clear signal and it will make a difference to the eventual outcome of the war, even if the ‘battle’ is stage managed.

I’d recommend union membership to anyone.  If you can join a union, you should.  Not only will they represent members’ interests collectively, they’ll also have your back if things go bad and make sure you get due process and fair treatment.  If I had a pound for every story I’ve heard about union representation and support making a real difference to how someone is treated, I’d make at least some of the cash back that I’ll lose by striking tomorrow.

Academics v. University administrators…. part 94…

A picture from the TV programme 'Yes Minister'This week’s Times Higher has another article about Benjamin Ginsberg’s book  The Fall of the Faculty: The Rise of the All-Administrative University and Why It Matters.  It’s written about the US, but it has obvious implications for the UK t00, where complaints from some academics about “bureaucrats” are far from uncommon.  Whether it’s that administrators are taking over, or that the tail is wagging the dog, or that we’re all too expensive/have too much power/are too numerous, such complaints are far from uncommon in the UK.

There’s two ways, I think, in which I would like to respond to Ginsberg and his ilk.  And it’s the “ilk” I’m more interested, as I haven’t read his book and don’t intend to.

The first way I could respond is to write a critical blog post, probably with at least one reference to the classic ‘what have the Romans ever done for us?‘ scene in Monty Python’s Life of Brian (“But apart from recruiting our students, hiring our researchers, fixing our computers, booking our conferences, balancing the books, and timetabling our classes, what have administrators ever done for us?”).  It would probably involve a kind of riposte-by-parody – there are plenty of things I could say about academics based upon stereotypes and a lack of understanding, insight, or empathy into what their roles actually entail.  Something about having summers off, being unable or unwilling or unable to complete even the most basic administrative tasks, being totally devoid of any common sense, rarely if ever turning up at work… etcetera and so on.  I might even be tempted to chuck in an anecdote or two, like the time when I had to explain to an absolutely furious Prof exactly why good governance meant that I wasn’t allowed to simply write a cheque – on demand – on the university’s behalf to anyone she chose to nominate.

The second way of responding is to consider whether Ginsberg and other critics might have a point.

On the whole, I don’t think they do, and I’ll say why later on.  But clearly, reading the views attributed to Ginsberg, some of the comments that I’ve heard over the years, and the kind of comments that get posted below articles like Paul Greatrix’s defence of “back office” staff (also in the Times Higher), there’s an awful lot of anger and resentment out there – barely constrained fury in some cases.  And rather than simply dismissing it, I think it’s worthwhile for non-academics to reflect on that anger, and to consider whether we’re guilty of any of the sins of which we’re accused.

I didn’t want to be a university administrator when I was growing up.  It’s something I fell into almost by accident.  I had decided against “progressing” my research from MPhil to PhD, because although  I was confident that I could complete a PhD (I passed my MPhil without corrections), I was much less confident about the job market.  Was I good enough to be an academic?  Maybe.  Did I want it enough?  No.  But it gave me a level of understanding and insight into – and a huge amount of respect for – those who did want it enough.  Two more years (at least) living like a student?  Being willing to up sticks and move to the other end of the country or the other side of the world for a ten month temporary contract?  Thanks, but not for me.  I was ready to move towards putting down roots.  I was all set to go off and start teacher training when a job at Keele University came up that caught my eye.  And that job was on what was then known as the “academic related” scale.  And that’s how I saw myself, and still do.  Academic related.

My point is, I didn’t sign up to be obstructive, to wield power over academics, to build an ’empire’, or – worst of all – to be a jobsworth.  I’ve never had a role where I’ve actually had formal authority over academics, but I have had roles where I’ve been responsible for setting up and running approval processes – for conference funding, for sabbatical leave, for the submission of research grant applications, and (at the moment) for ethical approval for research.  When I had managerial responsibility for an academic unit, my aim was for academics to do academic tasks, and for managers and administrators to do managerial/academic tasks.  That’s how I used to explain my former role – in terms of what tasks that previously fell to academics would now fall to me.   Nevertheless, academics were filling in forms and following administrative processes designed and implemented by me.  While that’s not power, it’s responsibility.  I’m giving them things to do which are only instrumentally related to their primary goal of research.  I am contributing to their administrative workload, and it’s down to me to make sure that anything I introduce is justified and proportionate, and that any systems I’m responsible for are as efficient as possible.

So when I hear complaints about ‘administration’ and ‘bureaucracy’ and university managers, whether those complaints are very specific or very general,  I hope I’ll always respond by questioning and checking what I do, and by at least being open to the possibility that the critics have a point.

However, I don’t think most of these complaints are aimed at the likes of me.  Partly because I’ve always had good feedback from academics (though what they say behind my back I have no idea….) but mainly because I’ve always been based in a School or Institute – I’ve never had a role in a central service department.  Thus my work tends to be more visible and more understood.  I have the opportunity to build relationships with academics because we interact on a variety of different issues on a semi-regular basis, which generally doesn’t happen for those based centrally.

And I think it’s those based centrally who usually get the worst flack in these kinds of debates.  I’m not immune from the odd grumble about central service departments myself in the past when I’ve not got what I wanted from them when I want it.  But if I’m honest, I have to accept that I don’t have a good understanding of what it is they do, what their priorities are, and what kinds of pressure they’re under.  And I try to remind myself of that.  I wonder how many people who posted critical comments on Paul’s article would actually be able to give a good account of what (say) the Registry actually does?  I would imagine that relatively few of the academic critics have very much experience of management at any level in a large and complex organisation.

I’m not sure, however, that all of the critics bother to remind themselves of this.  It’s similar to the kinds of complaints about the civil service and the public sector in general.  ‘Faceless bureaucrats’ is an interesting and revealing term – what it really means is that you, the critic, don’t know them and don’t know or understand what it is they do.  ‘Non-job’ is another favourite of mine.  There many sectors that I don’t understand. and which have job titles and job descriptions which make no sense to me, but I’m not so lacking on imagination or so arrogant to assume that that means that they’re “non-jobs”.  In fact, I’d say the belief that there are large groups of administrators – whether in universities or elsewhere – who exist only to make work for themselves and to expand their ’empire’, is a belief bordering on conspiracy theory.  Especially in the absence of evidence.  And extraordinary claims require extraordinary evidence.  That’s not to say that there is no scope for efficiencies, of course, but that’s a different scale of response entirely.

By all means, let’s make sure that non-academic staff keep a relentless focus on the core mission of the university.  Let’s question what we do, and consider how we could reduce the burden on academic staff, and be open to the possibility that the critics have a point.

But let’s not be too quick to denigrate what we don’t understand.  And let’s not mistake ‘Yes Prime Minster’ for a hard-hitting documentary….

Mind your PQQs…

A picture of Gollum from the recent Lord of the Rings films
"We wants it! We wants the precious tender documents!"

One of the things I do on a semi-regular basis in the search for research funding for colleagues is look at public sector tenders for possible opportunities.  The vast majority aren’t relevant for my purposes – I’m sure that tender for cadaver transport (presumably to remove the skeletons from the cupboards?) was a good opportunity for someone, though.

Something I’ve never understood – and if anyone can explain it I’d love to know – is why the UK public sector goes to such lengths to make it difficult to get hold of the full set of documents for their calls for tenders.  Usually there’s the briefest of summaries provided in a publicly accessible form, and it’s this that I usually send on.  To get the full picture, you need to log in to the relevant agency’s shiny web tendering system.

But you can’t log in until you’ve created an account.

And you can only create an account by giving full details of your organisation.  Who you are, where you are, what you do, what type of organisation you are, how many employees the organisation has.  Once that all been entered, then there are usually some activity codes to select to reflect the organisations main line of business.   No doubt this is all crushingly important – I wouldn’t want anyone to mistakenly think that the University of Nottingham was (a) an SME; or (b) a supplier of paperclip-related products.  It’s as if all these information requests are a precursor to a deep and meaningful long-standing relationship, when all I’m doing – for now – is having a quick flirt with a potentially attractive and available funding opportunity (“my academic mate might fancy you”) that may not prove to be our type.

I’m never sure whether to register as ‘University of Nottingham’ or ‘Nottingham University Business School’.  But the problem with signing up as Nottingham University Business School is that it doesn’t have a separate legal existence, while the problem with signing up as UoN is that no-one else can do so later.  I’ve had problems before when using a shiny electronic tendering system with trying to hunt down the institutional username and password set up as a result of a tentative interest in some other call which everyone’s long since forgotten about.

And… am I the institutional contact?  I can’t sign or submit stuff on behalf of the university, so… no.  However, I want the information, and I become very unpopular if I enter the details of the people who can sign stuff off so that they get bombarded with increasingly cryptic shiny electronic tendering system messages.

So far, so tedious.  I’ve put in the minimum possible level of information about UoN or NUBS, and I’ve taken a view on who ought to be the contact.  I’m now mildly annoyed.

Can I see the tender documents, now, please?

Well, no.  The system is going to email you first to confirm your account.  You’ll need to wait about ten minutes and then click on the link.  I’m now quite frustrated.

Okay, so now….

No.  You’ll need to log in again, using the username and password that we’ll send you, possibly in separate emails.  In about another ten minutes.  I’m now very annoyed, and I’m telling myself how more annoyed I’m going to be if this goes nowhere.

Right, so…

Ah, no.  You’ll have to change the password first.  Which means you need to think of a new one that’s memorable yet not important.  You might have to share it with others involved in any tender, so it shouldn’t be your internet banking password, your facebook page, your twitter account, your blog, or your university email password.

Then – and only then – can you see the precious documents.   Of course, if it turns out that the academic who asked you for your information turns out not to be interested, the price of the information you extracted is being bombarded with cryptic emails from the shiny system demanding that you log in… from now until the closing date.  And perhaps months later.  There’s nothing I find more helpful than being informed that some shiny electronic tendering system that I’d long forgotten about is being upgraded to an even shinier system and won’t be available from 7:30 to 7:32 on Sunday week.  I promise I’ll try not to let it spoil my weekend.

I understand why procurement people want to know who is interested in their tender.  I’d want to know that too.  I’d want to know if my call was reaching SMEs, or universities, or consultancies, or overseas, and so on.  I’d want to know who was interested, but didn’t tender, and perhaps even why.  But seriously…. does the process have to be so onerous?  Can’t the information just be made freely available – or at least available with a much shorter registration process?  I don’t want to register and set up a supplier profile on your shiny new system.

I.  Just.  Want.  The.  Call.  Information.  Please?