The American Institute of Biological Sciences (AIBS) has published findings from research it conducted on reviewer and applicant perceptions of criteria usage and risk evaluation in grant peer review.
These results, generated from a survey sent to over 13,000 scientists, suggest discrepancies between reviewers and applicants in terms of whether the reviews addressed innovation and risk; while most reviewers indicate these elements were essential and were factored into their ranking decisions, most applicants suggest these factors were not addressed in the feedback they received. Thus, these results suggest a potential source of bias in the review process.
As others have suggested, there may be different interpretations of excellence used by reviewers, as well as different risk tolerances; this is an area of future research. The manuscript, entitled “Risk evaluation in peer review of grant applications” was published online in late February, 2018 in the journal Environment Systems and Decisions.