Update on peer review for upcoming Project Grant competitions
Message from Adrian Mota, Associate Vice-President, Research Programs (Operations)
Dear colleagues,
On behalf of the Canadian Institutes of Health Research (CIHR), I thank you for your continued support in the face of the COVID-19 pandemic. Together, we have now delivered four Project Grant competitions using a virtual platform, including the competition now in progress. That you have managed to do so in addition to your research and caregiver duties and, in many cases, returning to campus to teach during these uncertain times, is a testament to your resilience and dedication to health research.
In keeping with CIHR’s goal of communicating openly with researchers, I want to share some updates regarding peer review for the Project Grant competition. I invite you to read on to learn more about our plans for the future and I wish to underscore that CIHR will continue to consult you on what these plans should look like.
Peer review for the Spring 2022 Project Grant competition
As the COVID-19 pandemic continues, it is clear that we are not out of the woods.
For that reason, I can confirm that the Spring 2022 Project Grant competition will be delivered virtually using MS Teams for peer review committee meetings.
To date, we have used MS Teams to deliver peer review for four Project Grant competitions, as well as our strategic funding competitions. Applicants and reviewers report that they have had mostly positive experiences with these virtual competitions. We will continue to make refinements where we can, to ensure that the reviewer experience runs smoothly and mimics the experience of face-to-face committee meetings as much as possible.
What’s on the horizon for peer review at CIHR?
We have gathered feedback from the community on their preferences for various models of peer review (i.e., face to face, fully using a virtual platform or a hybrid) in three separate surveys (of applicants to the Spring 2021 Project Grant competition, of reviewers for the Spring 2021 Project Grant competition, and of the wider community). The findings have been relatively consistent so far.
Preferred Model of Peer Review
Long description
Applicant
Total cases: 600
Total responses: 649
| Frequency | Cases (%) | Responses (%) | |
|---|---|---|---|
| Virtual peer review meetings only | 160 | 26.7 | 24.6 |
| Face-to-face peer review meetings when safe to do so | 320 | 53.3 | 49.3 |
| Hybrid approach (e.g.: one meeting per year held face to face and one meeting held virtually) | 169 | 28.2 | 26.0 |
Reviewer
| Total Responses | Virtual peer review meetings only | Face-to-face peer review meetings when safe to do so | Hybrid approach (e.g.: one meeting per year held face to face and one meeting held virtually) | Other | I prefer not to answer | ||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| What is your preferred model for the hosting of peer review meetings at CIHR post-COVID? | 288 | 63 | 21.88% | 148 | 51.39% | 67 | 23.26% | 5 | 1.71% | 5 | 1.74% |
Community
| Total responses | Virtual peer review meetings only | Face-to-face review meetings when safe to do so | Hybrid approach (e.g.: one meeting per year held face to face and one meeting held virtually) | Other | I prefer not to answer | |||||
|---|---|---|---|---|---|---|---|---|---|---|
| 1044 | 247 | 23.66% | 493 | 47.22% | 263 | 25.19% | 35 | 3.35% | 6 | 0.57% |
As you can see, the majority of our community prefers in-person, face-to-face peer review to other alternatives, either as a standalone process or some variant of a hybrid of virtual and face-to-face. However, a significant proportion of respondents favour a virtual model. What we do not know at the moment is whether one model or the other has preferential impacts on specific groups of reviewers, for example, whether those with young families or who are primary caregivers would potentially favour a virtual process. As an agency committed to making evidence-informed decisions, in our next phase of analysis we will seek to understand these proportions more deeply.
Beyond this, we know that each model has its strengths and weaknesses. For example, a virtual platform for peer review has enabled some researchers who previously would not have been able to travel to Ottawa for in-person meetings to participate in peer review. This has the potential to provide a more diverse slate of reviewers, and more international experts. In addition, at a time when we are all concerned about our collective impact on the environment, virtual platforms provide a meaningful opportunity to reduce our agency’s carbon footprint.
At the same time the sense of exhaustion following virtual meetings (Zoom fatigue) is a very real phenomenon that all of us have experienced. This is combined with the loss of many elements of the face-to-face review process such as the networking, mentoring, and camaraderie that can’t be replicated in a virtual format.
Given this, we commit to further studying different models (including those used by our international counterparts) to find the best options for our community. We will not drag this process out, and will soon be reaching out to you for further feedback on what those models might look like in practice. We will leverage groups such as the University Delegates Network and the Chairs/Scientific Officers to continue this conversation. Our focus, as ever, remains on creating processes that allow for the highest quality peer review.
I know that a lingering concern remains around the perception that virtual peer review results in a lower quality of discussion of applications. At the moment there is no evidence for this, but admittedly this question has not been closely examined. For that reason, we commit to studying the differences between face-to-face and virtual peer review processes and sharing more information as our work advances.
On the topic of peer review, we are also focused on ensuring that Project Grant peer review committee mandates are inclusive and relevant for the current research landscape. At the recent annual meeting of Committee Chairs and Scientific Officers, participants reviewed existing committee mandates and made suggestions for modifications and renewals. These discussions will continue with the goal of ensuring all submitted applications find appropriate homes for their review. We will share more information with you as this work progresses.
In summary, there are times when making few changes is the best course of action, and we have heard that the research community wants CIHR to deliver the Project Grant competitions reliably. CIHR cannot predict the future with any confidence, so we are taking things one step at a time, communicating via various channels (including Ask Me Anything sessions), and listening.
For now, I invite you to reach out to CIHR’s Contact Centre at support-soutien@cihr-irsc.gc.ca if you have any questions or feedback you wish to share. I wish you all good health and good luck.
Sincerely,
Adrian Mota
Associate Vice-President, Research Programs
- Date modified: