ERC funding and ERC evaluation

It was in 1990 as a young PhD student University of Cambridge when my supervisor Richard Jackson discussed with his friend Tim Hunt the funding situation in the UK, which saw a dwindling number of “alpha”-rated research proposals funded by the research councils (for example see correspondence in Nature Vol 348 by Michell). I still remember the incredulous comment by Richard to a news item about the low funding rates at the MRC: “…but this means there must be other criteria used than scientific excellence to come to the funding decision.” (which besides the point were apparently were not stated in the funders information). These were obviously very different times and researchers in the UK and all over the Western world quickly adapted to the new tunes, which were imposed by governments on national research funders under increasingly tight budgets. Whether it was the eventual medical use, the economic potential or societal impact, researchers in the US and Europe were forced to add a more or less lengthy section to their proposals about the potential outcomes of their research. In the framework programmes of the EU even an entire impact section became eventually necessary, which carries the same weight as the scientific excellence and implementation in the evaluation. Effectively reducing scientific excellence even in proposals for doctoral networks to just one of three decisive criteria.

“The sole criterion for selection is scientific excellence.”

When the European Research Council was founded to solve some problems and eventually revolutionise the European funding arena with their grants, the founders boldly stated their alternative concept for coming to funding decisions: “The sole criterion for selection is scientific excellence.” In my > 20 years of experience in the evaluation and writing of grants I believe this decision is at the heart of the robust and high quality of the evaluation process the ERC uses to select their grantees. While no system is fail-proof, the more than 50 applicants I have had the honour to support are testament to an excellent quality of evaluation on the basis of the profound feedback and assessments applicants receive. I would say throughout the years I have seen one decision amongst my customers I personally judge as at least debatable, where I presume two evaluators did not fully grasp the concept of the proposal. While this particular case still contains some particular sting, this is a stellar performance of the ERC. Compared to the comprehensive list of other funding regimes I have explored in my career as coach and ghost writer for research proposals, misunderstandings or even misgivings about the scientific concept are sadly rather the norm than the exception. Clearly the peer-review is reaching limits at some point and an effective moderation of the process by active researchers frequently fails to materialise in most national funding institutions. In my opinion, the high quality of the assessment by the ERC panels rests on several aspects of the evaluation, which are executed by a dedicated team of scientists and staff. Besides the high quality and commitment of the panels, I would like to highlight three procedural aspects of the ERC evaluation as critical elements:

• The focus on the top 20% of the applicants, which are selected in the first step, which focuses the effort of evaluation on the top level of applications.
• The decision to make funds available to select 10% of the applications in each panel.
• The personal interview of the applicants to provide amongst several other aspects the important opportunity to counter any potential criticism evaluators may have about the proposal.

Clearly, there are new challenges and criticism has been aimed at the ERC, including an uneven regional distribution of their grants and unfairly favouring those applicants from elite research institutions. It is therefore not only important to continue the development of the procedure, but also remain very open and transparent about the discussions. The ERC has an open process to continue to improve an already strong process and under president Maria Leptin the ERC has invigorated efforts recently to tackle challenges which are associated with peer review and the metrics used to define scientific excellence. Now she has commented on the recent changes in the evaluation procedure and proposal forms, explaining the rationale behind them and I recommend any applicant to read this document.

Importantly, The ERC as a funder of frontier research remains committed to retain the sole criterion of scientific excellence. As they rightly state any use of “economic or societal impact as explicit evaluation criteria would disfavour fundamental, curiosity-driven research that may not have an immediate or obvious economic or societal impact but is nevertheless important for scientific progress.” However, no outstanding researcher who regularly makes significant contributions to his/her field should now sit back and relax. Because as a potential applicant it is important to understand what the ERC defines as scientifically excellent and how it is measured. Obviously, different qualities are important when assessing grant proposals or deciding on say a faculty recruitment.

So what is scientific excellence according to the definition of the ERC?

According to the ERC, project proposals continue to be judged according to excellence in the ambition, potential scientific impact and scientific approach. Proposals should address important challenges, with a potential relevance beyond the immediate field and may arguably lead to major advances at the frontier of knowledge. ‘Ambitious’, ‘creative and original’ are descriptors for the kinds of proposals the ERC aims to fund. In this context, it is important to note that the ERC dropped the ‘high-risk, high-gain’ conjunction. As Maria Leptin states, it is not helpful for the evaluation of proposals, as it might principally disfavour applicants with strong preliminary data which show an exciting new approach is likely to work. Even though in practice this may not have been applied, it makes it even more important to abandon this ambiguous term to avoid confusion. Gone are the days I could watch the tongue-tied efforts of some of my customers to find a convincing answer to the question: “Why is your proposal high risk?”

The ERC has always put a strong emphasis on a match between the applicant and the proposal, which is also reflected in the focus on the interview as an important aspect of the evaluation. Leptin states a potential Matthew effect (Matthew effect – Wikipedia) may hamper the initial assessment of the proposal by the panel members, as project and applicant were scored in parallel and sums used to produce an initial ranking. To avoid an unfair bias favouring those applicants from elite research institutions the new regime now foresees to put a stronger emphasis on the evaluation of the project proposal, starting with the initial ranking of the applications.

The ERC will continue to set new standards in the evaluation of their proposals and clearly the narrative elements introduced with the recent changes along with the unusual format of the entire proposal will pose challenges to any applicant. Anyone embarking on this endeavour of competing for these prestigious grants faces the enormous challenge not only to develop a suitable research program, but also to present it in this very specific format. The support systems, which have sprung up around the ERC grants, as they can effectively make an academic career, provide prestige to the research institution and even are a matter of national pride are not helping to make the playing field more level. NCPs provide very diverse services of variable quality, some institutions mentor high potentials with strong expertise and dedicated support systems and some applicants have funds available to buy external support from coaches like me. Levelling the playing field for excellent researchers throughout Europe has not been resolved, despite some efforts recently introduced. However, the focus of the ERC to ensure a fair and effective evaluation of the proposals is an essential prerequisite. Therefore, the work of Maria Leptin and the entire team at the ERC is highly commended.

If you have questions about how to compete for ERC funding as a researcher or on an institutional level, or ERC evaluation processes feel free to contact me. We are very happy to engage researchers in particular from regions with less developed support systems and are happy to provide an initial assessment of your research plans without any obligation. Check the biolution website for more information https://biolution.net/proposal-support/