Results of survey on the future of peer review

Total responses: 1178

1. How would you primarily classify yourself?

• Early Career Researcher (ECR) (<5 years as an independent researcher; note: ECR status extended by 1 year)
• Mid-Career Researcher (5-15 years as an independent researcher)
• Senior Researcher (>15 years as an independent researcher)
• Knowledge User
• Other, please specify
• I prefer not to answer

Total responses: 1178

Researcher please specify:

Total responses: 33

2. What is your preferred official language of correspondence?

• English
• French
• I prefer not to answer

Total responses: 1180

In order to inform CIHR’s decision on how peer review is conducted post-COVID, we would like to hear your thoughts on various peer review models (e.g., in person face-to-face, virtual meetings that mimic the in person face-to-face process). Please complete the following questions:

3. Have you participated in any of the following virtual review processes?

• CIHR virtual peer review
• Virtual peer review for another agency
• None of the above

4. If you have participated to a CIHR virtual peer review process, have you been satisfied with your experience?

• Yes
  • Please indicate why
• No
  • Please indicate what CIHR could do to improve the virtual process
• I prefer not to answer

5. If you have participated to other agency (ies) virtual peer review process, have you been satisfied with your experience?

• Yes
  • Please indicate why
• No
  • Please indicate why
• I prefer not to answer

6. What is your preferred model for the hosting of peer review meetings at CIHR post-COVID?

• Virtual peer review meetings only
• Face-to-face peer review meetings when safe to do so
• Hybrid approach (e.g.: one meeting per year held face to face and one meeting held virtually)
• Other
• I prefer not to answer

7. Additional questions

• Please provide any feedback (pros and cons) for the virtual peer review model. If this was to be implemented, what key factors are necessary for the approach to be successful?
• Please provide any feedback (pros and cons) for the face-to-face peer review model. If this was to be implemented, what key factors are necessary for the approach to be successful?
• Please provide any feedback (pros and cons) for the hybrid peer review model. If this was to be implemented, what key factors are necessary for the approach to be successful?

Total responses: 627

1. How would you primarily classify yourself?

• Early Career Researcher (ECR) (<5 years as an independent researcher; note: ECR status extended by 1 year)
• Mid-Career Researcher (5-15 years as an independent researcher)
• Senior Researcher (>15 years as an independent researcher)
• Knowledge User
• Other, please specify
• I prefer not to answer

Total responses: 627

2. What is your preferred official language of correspondence?

• English
• French
• I prefer not to answer

Total responses: 627

3. What was your level of satisfaction with the application process in the Spring 2021 Project Grant competition:

Questions

• Overall Experience
• CIHR Contact Centre responsiveness to queries
• Effectiveness of CIHR communications
• Usefulness of Spring 2021 Project Grant Competition Webinars (“Ask Me Anything” and Question & Answer)
• ResearchNet system performance
• CIHR’s guidance on Integration of Sex and Gender Based Analysis
  

Answers

• Very Satisfied
• Satisfied
• Dissatisfied
• Not Satisfied
• Not Applicable

4. CIHR is considering changes to the appendices requirements (Other Application Material) and we appreciate your feedback on the following questions:

Questions

• Do you support the current appendices requirements (i.e.: unlimited ‘Other’ attachments with no obligation for the reviewers to read this material)?
• Do you support the current appendices requirements (i.e.: unlimited ‘Other’attachments with no obligation for the reviewers to read this material)?
• Do you support capping the number of attachments (e.g.: maximum of 2 document with page limits) with the requirement for reviewer read the material?

Answers

• Yes
• No
• Prefer not to answer

5. Based on feedback from the research community, CIHR implemented a number of changes to the Spring 2021 Project Grant competition. The following section asks about applicant use of and satisfaction with these changes.

Questions

• Clarity of instructions on the revised Summary of Progress
• Ease of use of the revised Summary of Progress
• Relevance of the revised Summary of Progress

Answers

• Very Satisfied
• Satisfied
• Dissatisfied
• Not Satisfied
• Not Applicable

6. Did you use the Applicant Profile CV?

Total responses: 616

Question

• Did you use the Applicant Profile CV
  • If so, please rate your level of satisfaction with the following elements:
    • Ease of use
    • Clarity of instructions

Answers

• Very satisfied
• Satisfied
• Dissatisfied
• Not Satisfied
• Not Applicable

7. Do you have any comments about the Applicant Profile CV?

8. Given the context of COVID-19 and the shift to virtual meetings, CIHR is would like to assess several models for the hosting of peer review meetings to inform our approach moving forward. As members of the research community, we appreciate your feedback on the following questions:

Questions: Virtual vs face-to-face experience

• What is your preferred model for the hosting of peer review meetings?
  • Virtual peer review meetings only
  • Face-to-face peer review meetings when safe to do so
  • Hybrid approach (e.g.: one meeting per year held face to face and one meeting held virtually)
• Please indicate why you selected this option:
• Do you have comments about the virtual-only model?
• Do you have comments about the face-to-face only model?
• Do you have comments about the hybrid model?
• Do you have any additional feedback or suggested process improvements for CIHR?

Applicant Peer Review Responses

Total cases: 600
Total responses: 649

1. How would you primarily classify yourself?

• Early Career Researcher (ECR) (<5 years as an independent researcher; note: ECR status extended by 1 year)
• Mid-Career Researcher (5-15 years as an independent researcher)
• Senior Researcher (>15 years as an independent researcher)
• Knowledge User
• Other, please specify
• I prefer not to answer

Total responses: 301

Total responses: 5

2. What is your preferred official language of correspondence?

• English
• French
• I prefer not to answer

Total responses: 301

3. Was this your first time reviewing for CIHR’s Project Grant competition?

• Yes
• No
• I prefer not to answer

Total responses: 297

4. Please indicate your level of satisfaction with your overall experience as a peer reviewer

Question

Please indicate your level of satisfaction with your overall experience as a peer reviewer

Answers

• Very Satisfied
• Satisfied
• Dissatisfied
• Not Satisfied
• Not Applicable

5. Did you attend the committee pre-meeting session?

• Yes
  • If yes, is there anything that you would improve, add or change?
• No
  • If not, please indicate the reason you could not attend, or if there was anything CIHR could have done to better accommodate your attendance?
• I prefer not to answer

Total responses: 297

6. Were you satisfied with the clarity of reviewer instruction provided in the reviewer manual?

• Yes
• No
  • If you were not satisfied, is there a specific section or set of instructions which you would suggest improving?
• I prefer not to answer

Total responses: 297

7. Were you satisfied with the overall preparation/training in the session provided through your interaction with CIHR staff, the reviewer manual, reviewer webinar and committee pre-meeting?

• Yes
• No
  • If not, please indicate what could be done to improve
• I prefer not to answer

Total responses: 297

8. Were you satisfied with the responsiveness to queries/concerns?

• Yes
• No
  • If not, please indicate what could be done to improve
• I prefer not to answer

Total responses: 297

9. Do you support the complete withdrawal of applications that do not follow the basic guidelines associated with the funding opportunity? This includes formatting (font size, spacing, margins, number of pages)?

• Yes
  • Please provide rationale for your answer
• No
  • Please provide rationale for your answer
• I prefer not to answer

Total responses: 292

10. As you know, a new Summary of Progress was added to the application process. Was this helpful  in your review?

• Yes
  • If yes, please explain why the summary of progress was useful in your review
• No
  • If no, please explain why the summary of progress was not useful in your review
• I prefer not to answer

Total responses: 292

11. What changes to the current use of attachments of other application material in the appendices would you support?

• Current format is fine – leave it to reviewers to access if necessary
• No appendices as the 10 page proposal should contain all relevant data
• Reduce the appendices from the current 5 to a lesser number
• I prefer not to answer

12. Would you agree to be a mentor for the Reviewer in Training program?

• Yes
  • If yes, please indicate why
• No
  • If no, please indicate if there is anything CIHR could do to improve to allow you to participate
• I prefer not to answer

Total responses: 292

13. Were you satisfied with the compensation for dependent care and other costs (if applicable) accrued as a result of participation in CIHR peer review?

• Yes
• No
  • If you were not satisfied, please indicate why
• Not applicable
• I prefer not to answer

Total responses: 296

14. Were you satisfied with how meetings were Chaired?

• Yes
• No
  • If you were not satisfied, please indicate why
• I prefer not to answer

Total responses: 296

15. Were you satisfied with the quality of Scientific Officer summaries offered at the meeting?

• Yes
• No
  • If you were not satisfied, please indicate why
• I prefer not to answer

Total responses: 296

16. In order to inform CIHR’s decision on how peer review is conducted post-COVID, we would like to hear your thoughts on various peer review models (e.g., in person face-to-face, virtual meetings that mimic the in person face-to-face process). Please complete the following questions;

• Have you been satisfied with your experience with CIHR virtual peer review processes to date?
• Yes
  • Please indicate why
• No
  • Please indicate what CIHR could do to improve the virtual process
• I prefer not to answer

Total responses: 288

17. Have you participated to other agency (ies) virtual peer review process?

• Yes
• No
• I prefer not to answer

Total responses: 288

• If yes, have you been satisfied with your experience?
  • Yes
    • Please indicate why
  • No
    • Please indicate why

Total responses: 165

18. What is your preferred model for the hosting of peer review meetings at CIHR post-COVID?

• Virtual peer review meetings only
• Face-to-face peer review meetings when safe to do so
• Hybrid approach (e.g.: one meeting per year held face to face and one meeting held virtually)
• Other (Please indicate why you selected this option.)

Total responses: 288

19. Remaining survey questions

• Please provide any feedback (pros and cons) for the virtual peer review model. If this was to be implemented, what key factors are necessary for the approach to be successful?
• Please provide any feedback (pros and cons) for the face-to-face peer review model. If this was to be implemented, what key factors are necessary for the approach to be successful?
• Please provide any feedback (pros and cons) for the hybrid peer review model. If this was to be implemented, what key factors are necessary for the approach to be successful?
• Do you have any additional feedback or suggested process improvements for CIHR?