A study of the influence of blind peer review on the nature and outcomes of funded research

CFA has been awarded a grant from the Villum Foundation to study the effects of a relatively new and novel grant form, Villum Experiment, that seeks to promote unorthodox and innovative research projects. Among the key features of the funding instrument, which was first awarded in 2017, are that applications are assessed through a blind review process where reviewers do not know the identity of applicants, that reviewers assess the applications independently of each other, and also have the option of choosing one application for support, regardless of scores by other reviewers (“golden ticket”).

The study will focus on:

1. what types of research the applicants applied for,
2. who received the grants,
3. how the research was carried out,
4. the results of the research.

For all of these four aspects, this grant could potentially have effects that differ from standard grant types based on conventional peer-review processes. And a key cross-cutting theme across these aspects is originality; i.e. to what extent does the Villum Experiment promote and generate original research?

The study will in particular seek to answer the following questions:

1.  Which applicants receive grants from Villum Experiment, and does the group of recipients deviate from what would be expected for a standard peer review process? Existing studies have highlighted in particular two forms of bias in standard peer review:
   1. Matthew Effects: funding tends to be awarded to the already successful;
   2. Gender bias: among researchers with equivalent qualifications or proposals, men are more likely to receive fundings.
2. Is the research funded by Villum Experiment less orthodox than research funded through standard grant types? Numerous aspects can be examined to address these questions, among these: the design and goals of the research proposals; how the research is conducted; and the originality of the research (both in relation to the grantees existing research publications and to research in the discipline more generally).