CIHR Reviewers' Guide for the Canada Postdoctoral Research Award (CPRA) program

Applications focused on global health, or which include international collaborations are eligible. Applications focused on the field of global health that demonstrate that the proposed research project has the potential to improve health in Canada and/or the global community are accepted and encouraged as more than ever before, the health of Canadians is deeply intertwined with the health of people around the world. Global health research is defined as any research with a focus on the health of people living in low- and middle-income countries, as well as research that aims to understand the systematic factors that shape health and are inherently global (e.g., globalization, equity, neglected diseases, and transnational risks). Global health is relevant to all pillars. Please consult International collaboration website and Global Health 3.0: CIHR's Framework for Action on Global Health Research for more information.

Knowledge translation (KT) is integral to CIHR's mandate and falls into two main categories: end of grant KT and integrated KT. With both categories of knowledge translation, CIHR expects researchers and trainees to disseminate their findings and facilitate their translation into improved health, more effective products or services, and/or a strengthened healthcare system.

For end of grant KT, many means of dissemination exist and the onus is on the researchers to select the most appropriate vehicle for the intended knowledge-user audience to ensure maximum impact. When the primary knowledge users are researchers, dissemination of results through the publication of high quality articles in accessible journals is appropriate, although other strategies that increase awareness of the results and facilitate their application may also be appropriate. When knowledge-user audiences outside the research community should be informed of specific research findings, dissemination plans with more ambitious goals and comprehensive strategies are expected. With integrated KT, stakeholders or potential knowledge users are engaged in the entire research process and the research is directed at producing solutions to issues or problems the stakeholders/knowledge users have identified.

CIHR believes that greater access to research publications and data will promote the ability of researchers and knowledge users in Canada and abroad to use and build on the knowledge needed to address significant health challenges. Open access will promote accessibility to CIHR-funded research and will serve to increase the international visibility of Canadian research. CIHR grant recipients are reminded to adhere to the responsibilities outlined in the Tri-Agency Open Access Policy on Publications. Under this policy, grant recipients must ensure that research papers and bio-molecular data generated from CIHR funding are freely accessible online.

CIHR expects that all applicants will integrate sex and gender into their research designs when appropriate. Sex- and gender-based analysis (SGBA) is an approach that systematically examines sex-based (biological) and gender-based (socio-cultural) differences between men, women, boys, girls, and gender-diverse people. The purpose of SGBA is to promote rigorous science that is sensitive to sex and gender and therefore has the potential to expand our understanding of health determinants for all people. Reviewers are asked to explicitly assess whether the integration of sex (as a biological variable) and/or gender (as a socio-cultural factor) is a strength, a weakness or not applicable to the proposal. Reviewers are also asked to comment on their assessment and to provide recommendations to the applicants on how they might improve the strength of their applications with respect to the integration of sex and/or gender. The SGBA section of the CIHR website provides helpful resources for applicants and peer reviewers alike, including CIHR's definitions for sex, gender, and SGBA, as well as information on applying SGBA to the development and assessment of research proposals. Reviewers must complete one of the Online Training Modules on Integrating Sex & Gender in Health Research linked to the Learning for participants in Peer Review section.

When evaluating applications that involve and engage with First Nations, Inuit and Métis Peoples, reviewers are strongly encouraged to review the following training module and literature:

• Background context on Indigenous Health Research
• Ethics of Health Research Involving First Nations, Inuit and Métis People
• First Nations: The OCAP® principles, widely known and accepted were developed by the First Nations Information Governance Centre
• Inuit: Inuit Nunangat Policy and Inuit Tapiriit Kanatami [ PDF (1.18 MB) - external link ] (ITK)'s National Inuit Strategy on Research (NISR) [ PDF (1.18 MB) - external link ] outlines data governance principles.
• Métis: principles guiding ethical research with Métis communities [ PDF (548 KB) - external link ]
• Visions for Distinctions-based Indigenous Health Legislation: Executive summary
• SSHRC Guidelines for the Merit Review of Indigenous Research

Reviewers are encouraged to adopt gender-neutral and gender-inclusive language in their reviews. In reviews, both written and spoken, gender-neutral and gender-inclusive language is more accurate and more respectful when discussing the science and the applicant.

When discussing the science remember that gender is non-binary. When research is meant to include all people, avoid binary statements like "men and women." Instead, consider phrases like "men, women, and gender-diverse people" or "people of all genders." In addition, be mindful of word choice. Below are a few examples of words that could be replaced with more gender-neutral and gender-inclusive terms:

• "Female/male anatomy" with specific anatomical language (e.g., ovaries, uterus, testes)
• "Mankind" with "humankind" or "people"
• "Man-made" with "machine-made", "synthetic" or "artificial"

When referring to the applicant use gender neutral pronouns or phrases. For example, use "they" or "the applicant," rather than "he" or "she". Remain mindful of word choice, as some words could be replaced with more gender-neutral and gender-inclusive terms:

• "Chairman" with "Chair" or "Chairperson"
• "Maternity/paternity leave" with "parental leave"

In order to ensure that positive measures are undertaken within CIHR's mandate to contribute to the vitality of English and French minority communities in Canada, CIHR:

• Encourages the appropriate inclusion of official language considerations of English and French speaking Canadians, including those living in linguistic minority communities, in health research design, conduct and application to improve health outcomes; and
• Facilitates equitable access to its programs and services for health research stakeholders in official language minority communities.

CIHR is a signatory of the San Francisco Declaration on Research Assessment (DORA), which "recognizes the need to improve the ways in which the outputs of scholarly research are evaluated". DORA recognizes that scholarly outputs include a broad range of contributions and impacts and are not limited to published journal articles.

Reviewers are to assess productivity and progress broadly by taking into consideration:

• a range of contributions (e.g., research publications, reports, books, guidelines, datasets, code, tools, training and mentorship, volunteerism, community engagement, standards, software and commercialized products) and impacts (e.g., influence on policy and practice, health outcomes, societal outcomes, and distinctions-based, meaningful and culturally safe health research), and
• the context of the applicant (e.g., leave history, career stage, area(s) of research, experiential knowledge, diverse career paths, family responsibilities, pandemic impact) and how it may have affected their progress.

Metrics such as number of publications and citations, and size/number of research grants should not be used in isolation to assess productivity. Reviewers should not use journal-based metrics (e.g., Journal Impact Factors) as surrogate measures of quality and impact of individual research publications. As stated in DORA, the "scientific content of a paper is much more important than publication metrics or the identity of the journal in which it was published".

Reviewers are strongly encouraged to review the list of resources available to them.

All documentation and information that CIHR entrusts to peer reviewers and observers must be maintained in strict confidence at all times. It must be used only for the purpose for which it was originally collected-namely, to review applications and make funding recommendations as applicable.

Guide on Handling Documents used in Peer Review

The use of online generative AI tools (such as ChatGPT) present risks to both applicants and reviewers. As set out in the Guidance on the Use of Artificial Intelligence, the use of AI in peer review constitutes of a breach of confidentiality when reviewers fail to safeguard applicant materials. Refer to CIHR Policy Statement for more details.

Confidentiality is information about a person that will not be disclosed directly or indirectly to anyone else without that person's prior expressed consent. The information provided by applicants in their applications is protected by the Privacy Act and is made available to external assessors for reviewing purposes only. The use of this information for any other purpose is a breach of the Privacy Act and could result in a CIHR investigation and/or report to the federal Privacy Commissioner's Office.

Committee members are not to approach or communicate in any way with applicants, or anyone outside of the committee, regarding any information relating to the review of a specific application or offer opinions on the applicant's chances of success. In turn, applicants are not to contact committee members, including the Chairs and Scientific Officers, regarding the status of their applications (for example, ratings).

By law, applicants have access to their own application files. Therefore, all written materials used in evaluating an application are made available to the applicants when they are notified of CIHR's funding decision for a competition.

CIHR makes every effort to ensure that its decisions are fair and objective by identifying and addressing any conflicts of interest between an applicant and a peer review committee member. According to the Conflict of Interest and Confidentiality Policy of the Federal Research Funding Organizations (COIC), a conflict of interest means a conflict between a participant's (e.g., committee members and observers) duties and responsibilities with regard to the review process, and a participant's private, professional, business or public interests.

A conflict of interest may be deemed to exist, or be perceived as such, when committee members and observers (if applicable):

• Are a relative or close friend, or have a personal relationship with the applicants;
• Are in a position to gain or lose financially/materially from the funding of the application;
• Have had long-standing scientific or personal differences with the applicants;
• Are currently affiliated with the applicants' institutions, organizations or companies^{Footnote 1};
• Are closely professionally affiliated with the applicants, as a result of having in the last six years:
  • Frequent and regular interactions with the applicants in the course of their duties at their department, institution, organization or company;
  • Been a supervisor or a trainee of the applicants;
  • Collaborated, published or shared funding with the applicants, or have plans to do so in the immediate future; or,
  • Been employed by the institution, when an institution is the applicant.
• Feel for any reason unable to provide an impartial review of the application.

All committee members are subject to the same conflict of interest guidelines. CIHR staff is responsible for resolving areas of uncertainty. All committee members must read and agree to abide by the COIC policy prior to viewing any application information.

Success of the peer review system is critically dependent upon the willingness and ability of all reviewers to:

• Exercise rigorous judgement;
• Be impartial and reasonable;
• Understand and take into account, in a balanced way the particular context of each application; and,
• Provide a constructive, quality review which helps the applicant by pointing out strengths and weaknesses that contributed to the application rating.

CIHR ensures transparency in the peer review process through several different mechanisms. All applications submitted to CIHR are evaluated by reviewers who provide an overall assessment of the application. This assessment is performed using the application requirements and review criteria established for each individual competition, as seen in the funding opportunity details. All reviews are shared with applicants.

On its website, CIHR publishes the Peer Review Committee Membership Lists and lists all CIHR successful applications by competition through its Funding Decisions Database.

To identify conflicts of interests on the assigned set of applications, reviewers are to follow these steps:

• Log into ResearchNet.
• On the home page, click on the link of their assigned committee to open the main task list.
• Complete the task "Review Confidentiality Policy and Conflict of Interest Guidelines" (once completed, it will "open" the other tasks).
  • Complete the Equity, Diversity and Inclusion Self-identification Questionnaire in ResearchNet;
• Open the "Manage Conflicts/Ability to Review" task.
  • For each assigned application, use the information provided to indicate if you are able to review or if there is a conflict. If there is a conflict, CIHR will reassign the application to another reviewer.

Notes:

1. CIHR assigns CPRA applications to one of two peer review committees based on the primary scientific area identified by the applicant. The committees are multidisciplinary and applications are not assigned to reviewers based on specific expertise. As such you are likely to be assigned applications that are not within your specific area of expertise. Reviewers are asked to assess applications from a generalist's perspective and should not declare conflicts simply because applications are outside their area of expertise. Although you may not be familiar with the field, we ask that each reviewer complete their reviews from the point of view of a generalist.
2. As other reviewers within your committee will also declare conflicts, it is likely that you will receive additional application assignments. The calibration of workloads will be maintained to ensure a fair peer review workload.

Once conflicts have been identified, the full content of remaining assigned applications will be available under the task "Conduct Reviews." Reviewers should then follow the steps below.

For security reasons, ResearchNet times-out rather frequently, and CIHR is unable to retrieve any lost information. Therefore, reviewers should save their work often so that it is not lost. Alternatively, reviewers can write their comments offline using the template as proposed in Appendix B and then copy and paste the text directly into the appropriate text box on ResearchNet when ready.

Also, as the reviewers perform their evaluation, the reviews can be saved as drafts by selecting "Save draft copy" on ResearchNet. This allows reviewers to make changes at a later time. However, in order to submit the written reviews and ratings to CIHR, reviewers must select "Submit Final Review". Afterwards, no modifications will be possible.

It is important for reviewers to respect the deadline provided by submitting their written reviews and scores via ResearchNet by the date specified via correspondence with CIHR staff responsible for this program. Delays in the peer review process will jeopardize CIHR's ability to release decisions to applicants by the published date. If, at any point in the process, a reviewer determines that they may not be able to submit their reviews on or before the deadline, they must contact CIHR staff as soon as possible.

Once all the overall weighted scores are submitted, CIHR performs a discrepancy review by calculating the final rating for each application. CIHR will then identify applications which are at risk of an unfair decision because of a wide spread between the reviewers' overall score. In such cases, CIHR will ask the assigned reviewers to reconsider their initial assessment and resubmit scores. To do so, they will be asked to get in contact with each other to discuss the application. Reviewers do not have access to the other reviewer's scores/reviews. However, when necessary, CIHR staff responsible for the program will help facilitate the re-review process.

For this purpose, it is recommended that reviewers keep their working notes on file until a few weeks after the competition results have been announced. After which, please delete and/or destroy any copies of the applications that you have printed, downloaded, or saved.