Is competitive research funding best?
Profiles of Norwegian science
Photo: Ilan Kelman
Is scientific peer review a combat sport?
Ilan Kelman
Agder, Norway
A good proportion of scientific research in Norway is completed through grants awarded by a competitive process. Researchers write applications, proposing what they wish to do. These applications are evaluated and ranked by other scientists, helping to determine who is given funding for what.
The idea is that only the best applications receive funding. In theory, poorly conceived ideas, unoriginal suggestions, and flawed methodologies are weeded out via the scrutiny of other scientists. In practice, does a competitive application process really produce better science?
The immediate flaw is that the total amount of funding is known in advance. Any program from the Research Council of Norway has a fixed budget with all applications to each program competing for the same pot.
If the money available can fund up to 10 projects, but 20 applications are given first-class ratings, then half of the top proposals will likely be declined. Or all projects could be funded with half their budget—hardly conducive to good science. Even worse, if 10 projects are to be funded, but those receiving top marks do not cover the budget allocation, then some second-rate proposals might go forward.
The Research Council of Norway continually promotes the need for increased research budgets by analyzing and critiquing the government’s approach and allocations. This balanced view notes the goods and bads of what research the government funds and how it is funded. How often is the base premise of competitive research funding interrogated?
Earlier this year, researchers published a paper doing exactly this. They examined how competitive funding affects the quality and efficiency of scientific research. Norway was one of the countries. The analysis demonstrates how competitive research funding does not necessarily lead to better science.
Instead, top-down research management and highly controlled research funding processes inhibit scientists in doing what they do best: original, creative, innovative, successful science. The impressive level of detailed analysis in this paper provides important, rigorous, verifiable evidence for the authors’ conclusion. The ultimate results are not surprising, given what we already know about the peer review process.
Even when reviewers are given the same criteria, the same instructions, and the same method for evaluating applications, qualitative and quantitative evaluations vary markedly. Meanwhile, nepotism and sexism are long-standing, substantiated concerns along with race and ethnic bias. Another analysis of multiple studies about peer review lamented the lack of evidence showing that peer review improves scientific projects or publications. Technical editing of manuscripts did contribute to improvements.
Given all the data identifying problems with competitive research funding, many scientists sighingly accept that winning a research grant has more to do with rolling the dice than with formulating a sparkling proposal. In fact, the extensive public money covering the time required to write and review applications detracts from doing science. There are even competitive grants available to fund the time and cost for writing grant applications!
Administrative processes are a huge burden too. Every grant application scheme requires an online system for submitting and managing the application. Typically, the administrative requirements for putting together an application take up about half the time of producing an application.
The overhead required to process applications, send them out for peer review, collate the comments, make decisions, and notify applicants removes resources from science. All this, according to the evidence, for little gain by improving science.
So why does Norway persist with the system of competitive research grants? Perhaps faith in the system, inertia, lack of better alternatives, or a combination. To paraphrase the oft-used maxim, it is the worst system available for science, apart from all the others.
Certainly, advocating for no scientific accountability would be as flawed. Meanwhile, user-driven science is important for researchers to ensure they serve society and that their science is used for policy and practice.
Could there be more fundamental and pernicious reasons for favoring the system of peer-reviewed grant applications? By issuing calls for proposals and assigning a budget to each call, the political masters have far more control over the scope of research which scientists in Norway have options to pursue. This does not dictate research topics directly. Instead, it limits the general areas to be investigated, forcing some to be out-of-bounds, while appearing to provide freedom and objectivity.
This, though, is a conspiratorial hypothesis. Determining the actual political control exerted over the Research Council of Norway and its decisions, and understanding the real reasons for supporting the infrastructure and processes of competitive research grants, would require detailed research. That is, a funded research project.
Would such a grant application pass peer review and be awarded by the very system it purports to critique?
This article originally appeared in the January 11, 2019, issue of The Norwegian American. To subscribe, visit SUBSCRIBE or call us at (206) 784-4617.
