Success rates explained

alex-hulkes-150Alex Hulkes is Strategic Lead for Insights at the ESRC, and is responsible for developing our ability to evaluate and carry out data-informed analysis of ESRC investments, policy and operation.

In his latest blog, he looks at new analysis on the relationship between success rates and decision volumes.

When writing A Brief History of Time Stephen Hawking was told that every equation he included in the book would halve its sales. So the two people who might actually read the most recent analysis of ESRC data (PDF) shouldn’t feel too bad.

The topic this time is the relationship between success rates and decision volumes. I must admit that the analyses on this topic published so far have not been quite right, but they’ve got better and this is now, as far as I can see, correct and the last word on the matter. But at the cost of an equation or two.

Actually the equation itself can be skipped if it’s not your sort of thing. The point is that it exists, it describes all manner of success rate data and that it allows us to say categorically (as far as p values allow us to) that ESRC success rates are positively associated with institutional decision volumes.

fig 1

We see a large range of decision volumes, and so can expect to see a wide range of institutional success rates. By accounting for this underlying relationship between the two we can identify under-and over-performing research organisations (ROs) more reliably. And it turns out that there are perhaps only two ROs which have genuinely high success rates. You’ll have to read the analysis to find out who they are.

Whenever I’ve mentioned this around the ESRC office the first response has always been ‘but won’t everyone just believe that they can increase their success rate by submitting more proposals?’. I doubt that many would actually believe this, and fewer would act on it. But in the interests of sector efficiency I should say that the analysis also explains why this response would be a bad idea. By all means learn from previous applications (it’s what I would do if I was in a university research office) and use that learning to inform decisions about what to submit, but do not just send more in.

Finally, returning to the book theme, if you would like to know nearly as much as we do about ESRC you should probably read this (PDF). Formally known as ‘The effects of ESRC’ but informally as ‘The big book of ESRC’, it’s recommended reading for anyone with an interest in what we do. Better still, it contains just one equation.


If you have any comments please email alex.hulkes@esrc.ac.uk

Visit the ESRC website for further details on our performance data, including demand management and grant processing.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s