Peer Review Manual – Project

Table of Contents

  1. Introduction
  2. Policies and Considerations Impacting Peer Review
  3. Principles of Peer Review
  4. Project Grant Application Review Process
  5. Peer Review Committees: Selection and Roles of Members

1. Introduction

The purpose of this manual is to provide information on the policies, procedures and processes for the peer review of applications submitted to the Project Grant competition, as well as to outline the roles and responsibilities of peer review committee members.

The Project Grant program is open to applicants in all areas of health research that are aligned with the CIHR mandate. It is designed to capture ideas with the greatest potential for important advances in fundamental or applied health-related knowledge, health care, health systems, and/or health outcomes, by supporting projects of research conducted by individual researchers or groups of researchers in all areas of health. The best ideas may stem from new, incremental, innovative, and/or high-risk lines of inquiry or knowledge translation approaches.

Project Grant applications follow a single-stage committee-based peer review process. This process involves the evaluation of applications by a group of reviewers, who have the required experience and expertise to assess the quality and potential impact of the proposed research and research related activities, within the context of the program’s objectives. These reviewers are grouped into Peer Review Committees based on their expertise and the topics of applications submitted to these committees.

Peer Review Committees (PRCs) are responsible for:

  • Evaluating individual applications;
  • Rating each application;
  • Discussing applications at the committee meeting; and,
  • Recommending a budget and term to support the proposed research if the application is approved.

Members of the Peer Review Committees should review the material in this manual as they prepare for and conduct evaluations of applications. It is essential that members of the committees read and become familiar with the Funding Opportunity. They are also expected to:

  • Become familiar with CIHR funding policies and guidelines as outlined in CIHR's Funding Policies;
  • Formally agree to abide by CIHR's Confidentiality and Conflict of Interest Policy through ResearchNet;
  • Ensure that all review materials are handled according to CIHR’s Guide on Handling Documents used in Peer Review;
  • Refer to the CIHR Application Administration Guide for detailed regulations on CIHR funding;
  • Complete training on the Project review process through online learning modules, including the unconscious bias module and attendance at webinars; and,
  • Support CIHR’s efforts to strengthen the integration of sex as a biological variable and gender as a social determinant of health in health research by:
    • completing one of the sex- and gender-based analysis training modules available online through the CIHR Institute of Gender and Health CIHR;
    • becoming familiar with the SGBA tools available to support peer review activities, as provided by CIHR; and
    • providing feedback on the integration of sex and/or gender into research proposals, where appropriate.

2. Policies and Considerations Impacting Peer Review

  • International and Global Health Research Collaborations
  • Knowledge Translation
  • Tri-Agency Open Access Policy on Publications
  • Sex- and Gender-Based Analysis (SGBA) and Health Research
  • Official Languages

2.1 International and Global Health Research Collaborations

Applications focused on global health, or which include international collaborations are eligible. Applications focused in the field of global health that demonstrate that the proposed research project has the potential to improve health in Canada and/or the global community are accepted and encouraged as more than ever before, the health of Canadians is deeply intertwined with the health of people around the world. Global health research is defined as any research with a focus on the health of people living in low- and middle-income countries, as well as research that aims to understand the systematic factors that shape health and are inherently global (e.g., globalization, equity, neglected diseases, and transnational risks). Global health is relevant to all pillars. Please consult International collaboration website and Global Health 3.0: CIHR’s Framework for Action on Global Health Research for more information.

2.2 Knowledge Translation

Knowledge translation (KT) is integral to CIHR's mandate and falls into two main categories, end of grant KT and integrated KT. With both categories of knowledge translation, CIHR expects researchers to disseminate their findings and facilitate their translation into improved health, more effective products or services, and/or a strengthened healthcare system. The costs of dissemination are eligible expenditures in all CIHR grants.

For end of grant KT, many means of dissemination exist and the onus is on the researchers to select the most appropriate vehicle for the intended knowledge-user audience to ensure maximum impact. When the primary knowledge users are researchers, dissemination of results through the publication of articles in high quality and accessible journals is appropriate, although other strategies that increase awareness of the results and facilitate their application may also be appropriate. When knowledge-user audiences outside the research community should be informed of specific research findings, dissemination plans with more ambitious goals and comprehensive strategies are expected. With integrated KT, stakeholders or potential knowledge users are engaged in the entire research process and the research is directed at producing solutions to issues or problems the stakeholders/knowledge users have identified.

2.3 Open Access Policy on Publications

CIHR believes that greater access to research publications and data will promote the ability of researchers and knowledge users in Canada and abroad to use and build on the knowledge needed to address significant health challenges. Open access will promote accessibility to CIHR-funded research and will serve to increase the international visibility of Canadian research. CIHR grant recipients are reminded to adhere to the responsibilities outlined in the Tri-Agency Open Access Policy on Publications. Under this policy, grant recipients must ensure that research papers and bio-molecular data generated from CIHR funding are freely accessible online.

2.4 Sex- and Gender-Based Analysis (SGBA) and Health Research

CIHR expects that all applicants will integrate sex and gender into their research designs when appropriate. Sex- and gender-based analysis (SGBA) is an approach that systematically examines sex-based (biological) and gender-based (socio-cultural) differences between men, women, boys, girls and gender-diverse people. The purpose of SGBA is to promote rigorous science that is sensitive to sex and gender and therefore has the potential to expand our understanding of health determinants for all people. Reviewers are asked to explicitly assess whether the integration of sex (as a biological variable) and/or gender (as a socio-cultural factor) is a strength, a weakness or not applicable to the proposal. Reviewers are also asked to comment on their assessment and to provide recommendations to the applicants on how they might improve the strength of their applications with respect to the integration of sex and/or gender. The SGBA section of the CIHR website provides helpful resources for applicants and peer reviewers alike, including CIHR's definitions for sex, gender and SGBA, as well as information on applying SGBA to the development and assessment of research proposals.

2.5 Official Languages

In order to ensure that positive measures are undertaken within CIHR's mandate to contribute to the vitality of English and French minority communities in Canada, CIHR:

  • Encourages the appropriate inclusion of Official Language considerations of English and French speaking Canadians, including those living in linguistic minority communities, in health research design, conduct and application to improve health outcomes; and
  • Facilitates equitable access to its programs and services for health research stakeholders in official language minority communities.

3. Principles of Peer Review

The integrity of the peer review process relies on well-established principles and policies that:

  • Ensure fair and effective evaluation;
  • Support CIHR’s mandate and objectives.

CIHR’s principles of review are:

3.1 Confidentiality

Confidentiality is information about a person that will not be disclosed directly or indirectly to anyone else without that person's prior expressed consent. The information provided by applicants in their applications is protected by the Privacy Act and is made available to external assessors for reviewing purposes only. The use of this information for any other purpose is a breach of the Privacy Act and could result in a CIHR investigation and/or report to the federal Privacy Commissioner's Office.

Committee members are not to approach or communicate in any way with applicants, or anyone outside of the committee, regarding any information relating to the review of a specific application, or offer opinions on the applicant’s chances of success or failure. In turn, applicants are not to contact committee members, including the Chairs and Scientific Officers, regarding the status of their applications (for example, ratings).

By law, applicants have access to their own application files. Therefore, all written materials used in evaluating an application are made available to the applicants when they are notified of CIHR's funding decision for a competition.

The identity of the reviewers will not be revealed to the applicants. However, a list of review committee members is typically published on the CIHR website 60 days after the approval of funding of a competition.

3.2 Conflict of Interest

CIHR makes every effort to ensure that its decisions are fair and objective by identifying and addressing any conflicts of interest between an applicant and a peer review committee member. According to the Conflict of Interest and Confidentiality Policy of the Federal Research Funding Organizations (COIC), a conflict of interest means a conflict between a participant's (e.g., committee members and observers) duties and responsibilities with regard to the review process, and a participant's private, professional, business or public interests.

A conflict of interest may be deemed to exist or be perceived as such when committee members and observers:

  • Are a relative or close friend, or have a personal relationship with the applicants;
  • Are in a position to gain or lose financially/materially from the funding of the application;
  • Have had long-standing scientific or personal differences with the applicants;
  • Are currently affiliated with the applicants' institutions, organizations or companies*;
  • Are closely professionally affiliated with the applicants, as a result of having in the last six years:
    • Frequent and regular interactions with the applicants in the course of their duties at their department, institution, organization or company;
    • Been a supervisor or a trainee of the applicants;
    • Collaborated, published or shared funding with the applicants, or have plans to do so in the immediate future; or,
    • Been employed by the institution, when an institution is the applicant.
  • Feel for any reason unable to provide an impartial review of the application.

*A reviewer is not automatically in conflict with an application if they are from the same institution as the applicant, but do not know or work with the applicant.

All committee members (Chairs, Scientific Officers, reviewers) and observers are subject to the same conflict of interest guidelines. CIHR staff is responsible for resolving areas of uncertainty. All committee members must read and agree to abide by the COIC policy prior to viewing any application information.

3.3 Fairness

Success of the peer review system is critically dependent upon the willingness and ability of all reviewers to:

  • Exercise rigorous judgement;
  • Be impartial and reasonable;
  • Understand and take into account in a balanced way the particular context of each application; and,
  • Provide a constructive, quality review which helps the applicant by pointing out strengths and weaknesses that contributed to the application rating.

3.4 Transparency

CIHR ensures transparency in the peer review process through a number of different mechanisms. All applications submitted to CIHR are evaluated by reviewers who provide an overall assessment of the application. This assessment is performed using the application requirements and review criteria established for each individual competition, as seen in the funding opportunity details. In addition, the Scientific Officer (SO) notes capture the committee discussion during the peer review deliberations. All reviews and SO notes, as applicable, are shared with applicants.

On its website, CIHR publishes the Peer Review Committee Membership Lists and lists all CIHR successful applications by competition through its Funding Decisions Database.

4. Project Grant Application Review Process

4.1 Assigning Applications

The competition process begins with the applicants submitting a registration followed by a full application. Applicants are required, within the ResearchNet registration process, to indicate their suggestion of up to two most relevant peer review committees and to provide a justification for the committee(s) selected. Suggested committee(s) along with the corresponding justification(s) must remain unchanged between registration and application. Following submission of registrations, the Chairs and Scientific Officers, together with CIHR staff, review the registrations assigned to their committees. Together, they are responsible for ensuring that their committees have the appropriate expertise and, upon accepting applications for review, accept responsibility for ensuring that their committees conduct a fair review. In some cases, applications may not meet a particular committee's mandate and may need to be reassigned. CIHR will make the final decision on which peer review committee will review each application based on the summary of proposed research received at registration. The final committee selected may not necessarily be the applicant’s first or second choice. Applicants will be informed of the peer review committee that reviewed their applications when they receive the Notice of Recommendation and Notice of Decision.

Following the submission of registrations, committee members are given access to the summaries of proposed research to declare any conflicts of interest and indicate their level of expertise (i.e. High/Medium/Low/Not enough expertise).

Chairs, Scientific Officers and CIHR staff then assign each application to three committee members based on optimal alignment between the application content and the reviewer’s declared expertise. Each reviewer will receive a maximum of 8 to 10 applications to review. Reviews can also be solicited from external referees when additional, specific expertise is required. Four (4) to five (5) weeks before the peer review committee meeting, all committee members, excluding external referees, are given access to the full applications assigned to their committee. External referees see only those applications assigned to them.

While Project has no limit on number of times an application can be resubmitted, the review of the application is conducted as per the evaluation criteria within a given competition. Every competition is distinct.

4.2 Reviewing Applications

4.2.1 Conducting reviews

Research proposals

Research proposals submitted to the Project Grant competition can be up to 10 pages for English applications and 12 pages for French applications*, should stand alone (i.e., should contain all the information required to support the research proposal), and should contain a complete description of the project. Figures, tables, charts and photographs can be submitted as supporting material.

* Evidence demonstrates that documents written in French require approximately 20% more space than similar documents in English.

Formatting

Although the applicants can present their research proposals in a free form format, the reviewers assess the application based on the specific adjudication criteria outlined in Section 4.2.2 below. In order to ensure that all applicants have exactly the same amount of space to write their research proposals, applicants must adhere to the formatting requirements.

Failure to comply with these requirements may negatively impact the evaluation/rating of the applications and can lead to their withdrawal from the Project Grant competition. CIHR expects that applicants will adhere to all applicable instructions and requirements. This includes font sizes, spacing, page limits etc. Should a reviewer see an application that may not have followed instructions, this should be brought to the attention of CIHR staff. In cases of non-compliance that result in extra pages being added by applicants, CIHR may reformat and remove any pages that exceed the stated limit with no further notification to the Nominated Principal Applicant.

Summary of Progress

Reviewers must also assess the Summary of Progress. This two-page document supports the research proposal by allowing applicants to:

  • Contextualize any results from research activities that support the current application;
  • Describe how the application fits within their overarching research program and why the requested funds are needed; and,
    • This should include a clear outline how the current budget request is distinct from funds currently held (as applicable) or overlaps and/or differs from applications submitted to other funding agencies/organizations (pending grants).
  • Indicate how COVID-19 has affected their research program.

Please note that the Summary of Progress is a narrative and not a detailed accounting of progress and funding. Details on funding can be found in the applicant’s CV and the Summary of Progress is complemented by other components of the application.

Response to previous reviews

If applicants are resubmitting a previously unsuccessful application, they may provide a response (up to 2 pages) to previous reviewers’ comments. Applicants who have uploaded a “response to previous reviews” must have included all the reviews received (and Scientific Officer notes if available) from a previous competition (either Project or another priority-driven initiative), in their entirety, in the response (the reviews do not count toward the 2 page response limit). Applicants do not have to respond to all the comments in the reviews, only those that are relevant to the revised application, however all reviews must be included in the application package.

Where an applicant does not include all the previous reviews in the application package as outlined above, reviewers are not obligated to read the “Response to Previous Reviews” and thus are not obligated to consider it in their assessment.

Reviewers are not to consider the previous ratings in the evaluation/rating of the current application. Applications should be evaluated/rated based on their content and not on their previous ratings.

CV(s)

Reviewers are asked to review the applicant(s) CV(s). Through their CVs, applicants highlight their recognitions, funding history, activities and contributions that best demonstrate their leadership, significant contributions and productivity in the context of their research field(s).

Academic applicants must continue to use their CIHR Biosketch CV. Non-academics, Indigenous organizations, knowledge users and international applicants have the option of uploading either the CIHR Biosketch CV or the new Applicant Profile CV as an attachment to the Participant Information.

Other Application Material (optional to review)

Additional attachments including letters of support, questionnaires, surveys, and up to five publications from the last five years, may be uploaded under Other Application Material task in ResearchNet as outlined in the Application instructions. However, reviewers are under no obligation to read Other Application Materials.

Must be Reviewed Optional to review: “Other Applications Materials”
  • Research proposal
  • Summary of Progress
  • Response to Previous Review (if applicable)
  • CV(s)
  • Appendices:
    • Letters of support
    • Consent forms
    • Questionnaires
    • Additional tables and figures
Completing the review in ResearchNet

For each application, reviewers will be asked to submit in ResearchNet:

  • A global rating using the descriptors and definitions on the rating scale (see below);
  • Top (competitive) or bottom (non-competitive) group selection;
  • A summary of the research proposal (in the reviewer’s own words);
  • A justification of the rating by stating the strengths and weaknesses of the project based on the evaluation criteria including the strengths and weaknesses of the sex and/or gender integration in the research proposal;
  • An additional separate comment on how to improve the integration of sex (as a biological variable) and/or gender (as a socio-cultural factor) in the research design, methods, analysis and interpretation, and/or dissemination of findings (when applicable);
  • A budget recommendation detailing whether the requested resources are appropriate to support the project as described in the application taking into account funds already held or pending as per the CV and Summary of Progress.

Reviewers should save their work often as ResearchNet times-out at short intervals (security), and CIHR is unable to retrieve any lost information. Alternatively, reviewers can write their comments offline in a Word document and then copy and paste the text directly into the appropriate text box on ResearchNet when ready.

Reviews must be written in the reviewer’s own words and must not be directly copied from elsewhere (including the application, comments from other reviewers or their own previous reviews). The opinions expressed in the reviews will be provided to the applicants without prior editing by CIHR. However, in circumstances where the Chairs, the Scientific Officers or CIHR deem that a review expresses comments that could be construed as sarcastic, flippant, arrogant, or inappropriate in any way or contains factually incorrect information regarding CIHR’s policies, CIHR reserves the right to remove such information from the review. A constructive, quality review is one that helps the applicant by pointing out the strengths and weaknesses of the research proposal.

The adjudication criteria, evaluation checkpoints, evaluation guidelines and other considerations regarding the sub-criteria are outlined in section 4.2.2 below.

Rating Scale

To ensure consistency, reviewers must adhere to the following rating scale:

Descriptor Range Definition

Outstanding

4.5 – 4.9

The application excels in most or all relevant aspects. Any short-comings are minimal.

If an application is innovative, fills an important critical gap in knowledge, has very few flaws, and the investigators are well poised to perform the research and have a very productive track record.

Excellent

4.0 – 4.4

The application excels in many relevant aspects, and reasonably addresses all others. Certain improvements are possible.

If an application is very interesting, makes important advances, the team is excellent, but there are some minor limitations that need to be addressed or a clear description of impact is missing.

Good

3.5 – 3.9

The application excels in some relevant aspects, and reasonably addresses all others. Some improvements are necessary.

If an application is compelling, but has limited scope or impact, and/or raised some concerns about the feasibility and/or team; or in other words, the grant has strengths, but needs work.

Fair

3.0 – 3.4

The application broadly addresses relevant aspects. Major revisions are required.

If an application has merits but also has many limitations. Will not be funded.

Poor

0.0 – 2.9

The application fails to provide convincing information and/or has serious inherent flaws or gaps.

If an application has significant flaws and is not ready to be funded. Will not be funded.

Summary of the Research Proposal

Reviewers provide a summary of the project to demonstrate their understanding of the research work that is being proposed. This summary must be written in the reviewer’s own words and must not be directly copied from the application, from previous reviews (including their own, if applicable), or from comments of other reviewers.

Rating

In ResearchNet, reviewers provide their overall initial rating for the application to one decimal place in advance of the peer review committee meeting. This initial rating should encompass the assessment of the application on the evaluation criteria. Note that reviewers are not bound by the initial rating and can change it during the peer review committee meeting.

Top (competitive)/Bottom (non-competitive) Selection

Reviewers also indicate if each of their assigned applications is competitive (top) or non-competitive (bottom) in its current form. The proportion of applications deemed competitive or non-competitive (top or bottom), may vary depending on the overall quality of the applications assigned to them for review.

Strength and Weaknesses of the Project

Reviewers highlight the strengths and weaknesses of the project, including the integration of sex and/or gender in the research proposal (when applicable). Reviewers are encouraged to provide strengths and weaknesses for each evaluation criterion; strengths and weaknesses that contributed to the application rating must be clearly articulated, as they will be used to:

  • Provide the other reviewers assigned to the application with a justification for the ratings given to the application; and,
  • Provide applicants with feedback

Budget Recommendation

  • Reviewers are required to verify the requested budget and justification. Reviewers may recommend that the budget remains as requested or recommend a reduction. Budget adjustment recommendations should be justified and be specific to item and amount.

For more information please see section 4.2.3 Budget Assessment.

Integration of Sex and/or Gender in the Research Proposal

While reviewers are expected to include comments on the overall integration of sex- and/or gender as part of the strengths and weaknesses of the proposal, reviewers are also asked to comment in a separate section on whether the integration of sex (as a biological variable) and/or gender (as a socio-cultural determinant of health) is a strength, a weakness, or not applicable to the proposal. Reviewers will also be asked to recommend to the applicants on how they might improve the strength of their applications with respect to the integration of sex and/or gender. Resources to support this assessment can be found on the CIHR website.

Other Considerations

Reviewers will also be asked to flag applications in ResearchNet that involve any of the issues outlined below. These issues are not to be considered in the rating of the applications unless they impact the scientific quality of the application, as detailed below:

  • Ethical and other policy requirements: Responsibility for ensuring that all research meets ethical standards is delegated by CIHR to the applicants’ institutions. Ethics forms are not required as part of the application. However, reviewers may flag specific issues, such as the use of human participants, animals, human tissues or hazardous material, or research that involve Indigenous Peoples, if they feel they have not been adequately addressed.
  • Human pluripotent stem cell research: Applications recommended for funding that involve the use of human stem cells must also be reviewed by the Stem Cell Oversight Committee (SCOC). Applicants are instructed to check the relevant box in the section entitled “Certification Requirements”. It is essential that the use of these stem cells be also assessed by reviewers.

The deadline for uploading the reviews to ResearchNet, along with the assessment of overall quality (top/bottom groups) and the initial ratings, is five business days before the committee meeting date. Once reviewers have submitted their reviews, they are able to see all reviews of the other reviewers assigned to the same applications. Once reviews are submitted, reviewers will no longer be able to modify them prior to the committee meeting. Reviewers who wish to revise their reviews after the committee meeting will have 5 business days to directly upload their changes (ratings and comments) to ResearchNet.

  • It is the responsibility of all peer reviewers to familiarize themselves in advance of the committee meeting with the applications to be assessed by their committee, as this will facilitate discussions at the face-to-face meeting.

4.2.2 Adjudication Criteria and Interpretation Guidelines

To interpret the adjudication criteria, it is important to keep in mind that the research proposal may exert only a basic/mechanistic impact, which is as important as the translational impact. The impact does not only mean near-future clinical relevance. Reviewers should evaluate whether the work proposed will significantly advance the proposed area of research.

Criterion 1. Concept

  • Significance and Impact of the Research

Criterion 2. Feasibility

  • Approaches and Methods
  • Expertise, Experience and Resources

Reviewers should take into consideration the career stage, research field, progress and productivity, and institution setting of the applicants when assessing each criterion. The evidence should be notable compared to peers in similar fields and career stages.

Notes:

  • Specific considerations should be taken into account in the review of applications submitted to the Indigenous Health Research (IHR) committee (see IHR Committee Considerations under each Adjudication Sub-criterion below).
  • Evaluation of all applications containing a Randomized Controlled Trial (RCT) as a major component will need to consider the specific RCT evaluation criteria in their assessment, irrespective of the assigned peer review committee (section 4.2.5 below).
  • Evaluation of applications reviewed in the Commercialization committee will need to include the assessment of both the Research and Technical plan and the Commercialization plan according to specific review criteria (section 4.2.6 below).
  • Specific considerations will be taken into account in the review of applications in the Tri-Agency Interdisciplinary Peer Review (TIR) committee. For the details on the peer review process and description of the evaluation criteria, please consult the Committee Peer Review Guide.
4.2.2.a Criterion 1: Concept
Significance and Impact of the Research

This sub-criterion is intended to assess the quality of what is being proposed. Each Evaluation Checkpoint is presented as a question and is followed by the Evaluation Guideline(s) for that Checkpoint.

  1. Is the project idea creative?
    • The project idea is among the best formulated ideas in its field, stemming from new, incremental, innovative, or high-risk lines of inquiry; new or adapted research in basic science, or health care, or health systems or health outcomes. When applicable, knowledge translation/commercialization approaches/methodologies should be considered, as well as opportunities to apply research findings nationally and internationally.
  2. Is the rationale of the project idea sound?
    • The project rationale is based on a logical integration of concepts.
  3. Are the overall goals and objectives of the project well-defined?
    • The overall goal and objectives of the project are well-defined and clear.
    • The goal states the purpose of the project, and what the project is ultimately expected to achieve.
    • The objectives clearly define the proposed lines of inquiry and/or activities required to meet the goal.
    • The proposed project outputs (i.e., the anticipated results of the project) are clearly described and aligned to the objectives.
  4. Are the anticipated project contributions likely to advance basic health-related knowledge, or health care, or health systems or health outcomes?
    • The context and needs (issues and/or gaps) of the project are clearly described.
    • The anticipated contribution(s) (e.g. publishing in peer-reviewed journals) are clearly described, and should be substantive and relevant in relation to the context of the issues or gaps.
    • The anticipated contribution(s) are realistic, i.e., directly stemming from the project outputs, as opposed to marginally related.
Considerations

This sub-criterion is not intended to assess feasibility of the project, expertise of the team or the potential of success. These will be assessed under Criterion #2: Feasibility

Research should focus on addressing an issue (e.g., hypothesis or question, problem, need or gap) in any area across the spectrum of health (basic biomedical, health-related knowledge, health care, health systems, and/or health outcomes).

Depending on the nature of the project, it may have a research and/or knowledge translation / commercialization focus. Also, depending on the nature of the project, the rationale may be well-supported by evidence (e.g., literature review, previous findings, environmental scan, market analysis, stakeholder or partner input). However, this level of justification is not required for all types of projects (e.g., high-risk lines of inquiry).

In cases where projects have a primary implementation, or knowledge translation / commercialization (application and uptake of research findings) focus, the importance of the research should be validated as being substantive and relevant by stakeholders and partners, i.e., by those who could directly benefit from, or make use of, the project outputs.

Indigenous Health Research (IHR) committee Considerations

The proposed research must be relevant to First Nations, Inuit and/or Métis priorities and have the potential to produce valued outcomes from the perspective of First Nations, Inuit and/or Métis participants and Indigenous Peoples more broadly.

Global Health Research

Projects that have a global health research focus, or include international collaborations, are eligible for support through the Project Grant program. CIHR welcomes all research, from fundamental to applied, with the potential to advance health-related knowledge, and/or improve health outcomes for Canadians and/or the broader global community.

4.2.2.b Criterion 2: Feasibility
4.2.2.b-1 Approaches and Methods

This sub-criterion is intended to assess the quality of the project's design and plan; including how and when the project will be completed. Each Evaluation Checkpoint is presented as a question and is followed by the Evaluation Guideline(s) for that Checkpoint.

  1. Are the approaches and methods appropriate to deliver the proposed output(s) and achieve the proposed contribution(s) to advancing health-related knowledge, health care, health systems, and/or health outcomes?
    • The research and/or knowledge translation/commercialization approaches, methods and/or strategies are well defined and justified in terms of being appropriate to accomplish the objectives of the project.
    • Is sex (as a biological variable) and/or gender (as a socio-cultural factor) taken into account in the research design, methods, analysis and interpretation, and/or dissemination of findings?
    • Opportunities to maximize project contributions to advance health-related knowledge, health care, health systems and/or health outcomes should be proactively sought and planned for, but may also arise unexpectedly.
  2. Are the timelines and related deliverables of the project realistic?
    • Timelines for the project should be appropriate in relation to the proposed project activities. Key milestones and deliverables should be aligned with the objectives of the project, and be feasible given the duration of the project.
  3. Does the proposal identify potential challenges and appropriate mitigation strategies?
    • Critical scientific, technical, or organizational challenges should be identified, and a realistic plan to tackle these potential risks should be described. An exhaustive list is not expected.

Sex and Gender Considerations (if applicable)

Evidence demonstrates that biological and social differences between women and men contribute to differences in health risks, health services use, health system interaction and health outcomes. Accounting for sex and gender in health research has the potential to make health research more rigorous, more reproducible, and more widely applicable. CIHR expects that all research applicants will integrate sex and gender into their research designs when appropriate, as indicated on the Sex, Gender and Health Research webpage. Resources to assist reviewers in their assessment of the integration of sex and gender in the research design are available on CIHR’s website.

Indigenous Health Research (IHR) committee Considerations

In addition to demonstrating scientific excellence (non-Indigenous, Indigenous, or both), the proposed research approaches and methods must respect Indigenous values and ways of knowing and sharing, and abide by the Tri-Council Policy Statement Chapter 9: Research Involving the First Nations, Inuit and Métis Peoples of Canada, and/or Indigenous partnering community / organizational ethical guidelines, or clearly explain why other guidelines have been developed and agreed upon with the study governance body.

Other Considerations

Applications submitted to the Project Grant competition may include an integrated knowledge translation approach or may have a knowledge translation focus, with at least one knowledge-user and one researcher. CIHR defines a knowledge user as an individual who is likely to be able to use the knowledge generated through research to make informed decisions about health policies, programs and/or practices. A knowledge user can be, but is not limited to, a practitioner, policy-maker, educator, decision-maker, health care administrator, community leader, or an individual in a health charity, patient group, private sector organization or media outlet.

CIHR defines integrated knowledge translation as a way of doing research with researchers and knowledge users working together to shape the research process – starting with collaborations on setting the research questions, deciding the methodology, being involved in data collection and tools development, interpreting the findings and helping disseminate the research results.

Designs, approaches, and methodologies will vary by project (e.g., field of research, target audience) and should include a knowledge translation approach, when applicable, that is appropriate to the nature of the project outputs.

4.2.2.b-2 Expertise, Experience and Resources

This sub-criterion is intended to assess the appropriateness of the complement of expertise, experience and resources among the applicants (Nominated Principal Applicant, Principal Applicant(s) and Co-Applicant(s)), and their institutions/organizations, as it relates to the ability to collectively deliver on the objectives of the project.

Each Evaluation Checkpoint is presented as a question and is followed by the Evaluation Guideline(s) for that Checkpoint.

  1. Does the applicant(s) bring the appropriate expertise and experience to lead and deliver the proposed output(s), and to achieve the proposed contribution(s)?
    • The applicant(s) should demonstrate the combined expertise and experience needed to execute the project (i.e., deliver the proposed outputs as well as achieve the proposed contribution(s)).
    • The roles and responsibilities of each applicant should be clearly described, and linked to the objectives of the project.
  2. Is there an appropriate level of engagement and/or commitment from the applicant(s)?
    • The level of engagement (e.g., time and other commitments) of each applicant should be appropriate to the roles and responsibilities described.
  3. Is the environment (academic institution and/or other organization) appropriate to enable the conduct and success of the project?
    • Project applicants should have access to the appropriate infrastructure, facilities, support personnel, equipment, and/or supplies to:
      • Carry out their respective roles; and,
      • As a collective, manage and deliver the proposed output(s), and achieve the proposed contribution(s).
  4. Does the applicant adequately demonstrate productivity and progress of their research program?
    • In their Summary of Progress, the applicant should:
      • Outline the most relevant accomplishments; and
      • Demonstrate their productivity.

Reviewers must assess productivity broadly (i.e., not just based on publications) and consider the applicant’s context (e.g., career stage, leave history). CIHR has signed San Francisco Declaration on Research Assessment (DORA), which recognizes that scholarly outputs are not limited to published journal articles but can include a broader range of outputs. Reviewers are encouraged to include these in their assessments.

Indigenous Health Research (IHR) Committee Considerations

Appropriateness of the team based on their overall scientific experience (non-Indigenous, Indigenous, or both) and skills as well as their Indigenous community-based research experience, track record, relevance of past experience, including expertise related to Indigenous Health Research.

Other Considerations

The required complement of expertise will vary by project. Applications with an integrated knowledge translation approach or knowledge translation focus must include knowledge users in defining/refining research questions, informing the research plan, conducting research, interpreting research findings, understanding the receptor community, leading dissemination activities, etc. Knowledge users may also be responsible, and accountable for the application/uptake of the project outputs. The nature, breadth and depth of the applicants’ experiences and contributions should be assessed in the context of the applicants’ career stages.

Applicants that have taken leaves of absence in the past seven years (e.g., parental, bereavement, medical, or administrative leave) may include a PDF document (no page limits) to supplement the publication information for that equivalent period of time. Whatever length of time an applicant has taken off from research in the past seven years is the amount of time that they may include in the attachment. Note that leaves of absence should also have been included in the appropriate section of the CV. Reviewers should therefore review this document in order to ensure that they have an accurate profile of applicants’ research activities and achievements.

Project environments should be assessed according to their ability to support the proposed project activities. Institutions often function as "networked" environments or interdisciplinary networks, which means there may be multiple satellite environments contributing to the support environment. Reviewers should consider that for smaller institutions, or affiliated research facilities where resources and/or services may be obtained through networks, or may be contracted out.

4.2.3 Budget Assessment

The budget assessment must not be factored into the scientific assessment and must not influence the rating of applications.

While budget recommendations are not factored into the overall rating of an application, CIHR values the experience and perspectives of reviewers in estimating overall budget requirements of the proposed project.

Applicants are asked to submit their budget using a modular template with pre-set incremental levels, and to provide a comprehensive description of what the funds will be used for, in order to justify the amount requested. It is incumbent on the applicant to illustrate clearly to reviewers why the requested funds are needed and how they are distinct from funds currently held.

Reviewers are required to check the requested budget and justification although detailed scrutiny of each item is not expected. Reviewers are required to determine if the requested budget is appropriate to support the proposed project and if it is realistic and well justified in relation to the applicant’s current funding. Appropriate budget planning should be demonstrated. The requested resources, together with any existing resources, should be adequate to financially support the full scope of the project.

Partner contributions, if applicable (required for partnered applications), should be considered when reviewing support for the proposed project. Partner contributions will be outlined in letters of contribution and in the budget module.

Reviewers may recommend that the budget remains as requested or recommend a reduction. If a reviewer suggests an adjustment to the budget, he/she will be required to provide comments in the “Budget” section of their reviews, to justify their recommendation.

Each Evaluation Checkpoint is presented as a question and is followed by the Evaluation Guideline(s) for that Checkpoint.

  1. Is the requested funding appropriate to support the project? Is it realistic and well justified?
    • Appropriate budget planning should be demonstrated. The requested resources should be adequate to financially support the full scope of the project.
    • Reviewers are asked to consider the proposed budget of the project, and to provide a justified recommendation as to whether the budget should be:
      • Accepted, as described; or,
      • Reduced by a specific dollar amount (total).
  2. Is there any overlap in the applicant’s currently held funding and the current application?
    • Consider other sources of funds applied for or received
    • Is the funding requested distinct from that currently held?
    • Is the rationale for needed funding clear and complete?

4.2.4. Indigenous Health Research (IHR)

Applications with a central focus on carrying out meaningful and culturally safe research involving Indigenous Peoples, with the intent to promote health through research that is in keeping with Indigenous values and traditions and follows the TCPS 2 - Chapter 9 Research Involving the First Nations, Inuit and Métis Peoples of Canada guidelines may be reviewed by the Indigenous Health Research (IHR) committee. A significant portion of the IHR committee will be composed of reviewers with living-experience, knowledge, and/or expertise in Indigenous Health Research. Reviewers will provide their written reviews with a mentorship lens.

At the time of application submission, the research proposal must also explicitly describe engagement with the community in relation to the research. The IHR committee will take specific considerations into account when evaluating applications (see IHR Committee Considerations under each Adjudication Sub-criterion above). Applications that do not fit with the IHR committee mandate will be reviewed by another discipline-based committee.

All applications will be discussed at the committee meeting to allow for the development of an explicit mentoring approach to enable all promising Indigenous health research to continue to evolve.

The IHR committee may assess an application as being eligible for the Iterative Peer Review Process designed in consultation with the Reference Group on Appropriate Review Practices for Indigenous Health Research. The objective of the Iterative Peer Review Process is to allow applicants whose applications have been deemed excellent, the opportunity to provide minor clarifications that would see the application improve to become outstanding. Additional information can be found in the TCPS 2 - Chapter 9 Research Involving the First Nations, Inuit and Métis Peoples of Canada guidelines.

4.2.5 Randomized Controlled Trials (RCTs)

A randomized controlled trial (RCT) is an experiment in which investigators randomly assign eligible human subjects or other units of study (e.g., classrooms, clinics, playgrounds) into groups to receive or not receive one or more interventions that are being compared. The results are analyzed by comparing outcomes in the groups. Applications involving a RCT are accepted and encouraged where appropriate based on scientific and methodological grounds.

Irrespective of the suggested peer review committee, evaluation of all applications containing a RCT as a major component will need to consider the specific RCT evaluation criteria. Please consult the RCT Evaluation Criteria and Headings website for detailed information on the specific criteria that need to be taken into account by the committees when evaluating RCT applications.

4.2.6 Commercialization Projects

Commercialization projects are designed to advance discoveries/inventions towards commercializable technologies/products, with a view to attract new investment, create new science-based businesses, organizations and initiatives, and ultimately improve health outcomes for Canadians.

Please consult the Commercialization Projects website for detailed information on the specific criteria that need to be taken into account by the Commercialization committee when evaluating applications.

4.2.7 Tri-Agency Interdisciplinary Peer Review (TIR) committee

A new Tri-Agency interdisciplinary Peer Review (TIR) committee has been added as part of the tri-agency response to recommendations made in the Fundamental Science Review [ PDF (7.91 MB) - external link ] to ensure the agencies have robust mechanisms to review and support interdisciplinary research. While the three funding opportunities (Insight Grant, Project Grant, Discovery Horizons Grant) have distinct timelines and processes, this pilot will bring relevant applications together in one shared interdisciplinary peer review committee. As such, applications reviewed in the TIR committee will undergo a unique peer review process, and be evaluated against specific evaluation criteria (please see Section 5. Description of Evaluation Criteria of peer review guide) within the context of the Project Grant Adjudication Criteria and Interpretation Guidelines. TIR committee members are asked to familiarize themselves with the TIR peer review guide. Customized training will also be provided ahead of undertaking the peer review task.

4.3 The Peer Review Committee Meeting

The prime responsibility of a peer review committee is the evaluation of applications submitted to a competition by using CIHR's rating scale. The committee will also recommend the grant term and a budget sufficient to support the full proposed research. It is important that committees follow defined procedures in order to function in a consistent manner.

Of note, any committee member who has a conflict of interest with an application (as defined in the section 3.2 above) must not take part in the evaluation process of that application and must leave the meeting room before the application is discussed during the committee meeting. The Chair and CIHR staff are responsible for monitoring conflicts and for resolving areas of uncertainty.

4.3.1 Streamlining of Applications

Only a portion of the Project Grant applications submitted to a competition will ultimately be funded. Therefore, it is important that during the face-to-face meeting, committee members focus their discussions on the most competitive applications to ensure and achieve an accurate comparative evaluation. To help support this goal, a streamlining process is used to remove applications that are not competitive from the discussion process, thereby allowing committee members more time to judge and discriminate between potentially successful applications and helping to ensure that the most deserving applications receive funding. Applicants whose proposals are streamlined still benefit from the review process, as they receive the written reviews from the assigned reviewers (but no detailed Scientific Officer notes).

Applications that are streamlined will not be eligible to receive funding from the CIHR Priority Announcements (PAs). For more information on Priority Announcements please refer to the most recent Project Grant: Priority Announcement (Specific Research Areas) funding opportunity.

Procedure
  1. Prior to the committee meeting

    Reviewers provide their initial ratings on ResearchNet in advance of the committee meeting. They also indicate if each of their assigned applications is competitive (top) or non-competitive (bottom) in its current form. The applications in the competitive (top) group should include those applications considered to be the most deserving of being funded. The proportion of applications deemed competitive or non-competitive (top or bottom), may vary depending on the overall quality of the applications reviewed. Averages of the initial scores and top/bottom are recorded by CIHR staff and provided to the committee Chair and Scientific Officers. This information is used to determine which applications are eligible for streamlining.

    An application is streamlined if it meets the following conditions:

    1. The average of the reviewers’ scores places the application in the bottom 50% of all applications in the committee;
    2. At least one reviewer identified the application as non-competitive (bottom); and,
    3. There is no objection from any committee member to streamline the application.
  2. At the committee meeting

    All the committee members will be informed of the applications identified prior to the committee meeting as eligible for streamlining. If there are no objections from any committee members, these applications will be streamlined. If there is an objection to an application being streamlined, it will be discussed by the committee. Note that some of the streamlined applications may have an initial average rating above 3.5.

    During the course of the committee meeting, additional applications may also be identified as eligible for streamlining.

    If an application is not discussed, the applicant will receive all the reviewers' reports, but the Scientific Officer notes will only carry notification of the decision to streamline. Committee members do not vote on the rating of a streamlined application; the final rating is calculated as the average of the ratings from the assigned reviewers.

4.3.2 Assessment of Applications

The assessment of applications begins with the assigned reviewers announcing their initial ratings to one decimal place. The reviewers may revise their initial ratings in light of the committee discussion.

If an application is not streamlined, the committee discussion proceeds as follows:

  • The primary reviewer gives a brief synopsis of the proposal and presents his/her assessment, describing strengths and weaknesses of the proposal (see section 4.2.2 above for the adjudication criteria) including how the applicant(s) have addressed the sex and/or gender considerations;
  • The two secondary reviewers follow, focusing on points of agreement or disagreement with the other reviewers, and elaborating on points not already addressed;
  • If external reviews have been provided by external referees, the reviewers read the reviews;
  • The Chair opens the discussion to all committee members;
  • The Scientific Officer reads back the Scientific Officer notes to the committee, capturing the key elements of the committee discussion to be considered by the committee when rating the application; and,
  • The Chair seeks a consensus rating from the three reviewers. If a consensus cannot be reached, the Chair determines the consensus rating, usually by using the mean value of the reviewers’ ratings after committee discussion (round up, if necessary, to obtain a single decimal point).

4.3.3 Voting

All the committee members not in conflict (excluding the Chair and Scientific Officers) are asked to vote (+/- 0.5 of the consensus rating) for the final rating of each application.

4.3.4 Budget and Term

The budget requested will not be factored into the scientific assessment and rating of the applications. However, CIHR will seek the recommendation from the reviewers on the budgets and terms requested. For additional information, please see section 4.2.3 above.

CIHR reserves the right to determine the final amount awarded.

4.3.5 Flagging of Applications

Issues related to ethics, eligibility, human pluripotent stem cell research and other concerns need to be flagged for CIHR's attention. The rating should not be influenced by such factors. For additional information, please see section 4.2.

4.3.6 End of Meeting Review

A committee debrief provides an opportunity for CIHR staff to address any concerns of the committee members and to record feedback on the peer review process as part of CIHR's ongoing efforts to maintain an effective and high quality peer review system.

4.4 After the Committee Meeting

Upon completion of the committee meeting, for all applications discussed, CIHR will collect the final ratings and recommendations on funding level and grant term from all the committee members. This information will be used for funding decision-making.

All applicants will first receive a Notice of Recommendation, along with a copy of all reviews and the Scientific Officer notes (for the applications discussed by the committee). After, a letter of decision and a Notice of Decision will be sent indicating whether or not their proposals were approved, and if approved, with information on the allocated funds.

Applications that have been flagged for special attention for follow-up by CIHR staff are marked as "pending" until the applicants provide required information.

5. Peer Review Committees: Selection and Roles of Members

5.1 Selection of Peer Review Committee Members

Invitations to members of the health research community to join specific Project Grant Peer Review Committees (PRCs) are based on their area(s) of expertise, experience in grant writing, peer review process, and in consultation with the committee Chairs and Scientific Officers. The Chairs also have a role in the selection of Scientific Officers.

5.2 Committee Memberships

Project Grant Peer Review Committees consist of a Chair, two Scientific Officers and reviewers. The committee membership as a whole considers one or more of the following aspects:

  • The need to cover the full range of relevant research areas, methodologies and experience according to the committee’s defined mandate);
  • The necessity for reviewing capability in English and French so that applications in either official language can be evaluated by the committee; and/or
  • The need for national representation and representation by gender proportionate to the Canadian health research community.

5.3 Committee Members Roles and Responsibilities

5.3.1 Committee Chair

The committee Chair is expected to follow a clear set of general instructions as defined by CIHR.

Before the committee meeting, the Chair will:

  • Suggests names of potential committee members (Scientific Officers and reviewers) to CIHR during both the formation of the review committee and the assignment of applications, as required; this could also include identifying potential external referees to fill expertise gaps;
  • Works with the Scientific Officers and CIHR staff to assign applications to specific peer review committees and to select reviewers for each application;
  • Works with CIHR staff to establish a review sequence or a meeting agenda, and to identify the eligible applications for streamlining.; and,
  • Works with CIHR staff to manage conflicts of interest of committee members.

During the committee meeting and in the role of moderator, it is the Chair’s responsibility, in collaboration with CIHR staff, to ensure that the review committee functions smoothly, effectively and objectively, and that a positive, constructive, fair-minded environment in which research proposals are evaluated is established and maintained. The Chair will:

  • Provide opening remarks to the committee;
  • Explain the meeting process to the committee;
  • Ensure that all committee members who are in conflict with an application leave the meeting room before the discussion of the application;
  • Appoint a delegate as Chair or Scientific Officer when either individual leaves the meeting room due to a conflict of interest with an application or for any other reasons;
  • Fulfill an oversight role – does not rate applications nor vote during the committee meeting;
  • Ensure the involvement of the entire committee in evaluating each application;
  • Work with the Scientific Officers, as required, to summarize the discussion around each application, before the consensus rating is reached;
  • Ensure that a consensus rating is reached by the assigned reviewers;
  • Ensure that both the budget and term of the project are discussed by the committee and that comments are added to the Scientific Officer notes, as appropriate; and,
  • Ensure that specific ethical concerns and other CIHR requirements are addressed, and that any related discussion is captured in the Scientific Officer notes.

At the end of the committee meeting, the Chair may:

  • Propose that any application that has been identified by committee members as being inconsistently reviewed during the meeting is discussed again; and/or
  • Allow time to receive feedback from the committee members regarding the effectiveness and functioning of the committee.

5.3.2 Scientific Officers

Scientific Officers (SOs) are responsible for supporting the Chair in his/her role during the committee meeting and are expected to follow a clear set of general instructions as defined by CIHR.

Before the committee meeting, the Scientific Officers will:

  • Suggest names of potential committee reviewers to CIHR during the formation of the review committee, as well as during the assignment of applications, as required; this could include identifying potential reviewers to fill specific expertise gaps;
  • Work with the Chair and CIHR staff to assign applications to specific peer review committees and to select reviewers for each application;
  • Work with the Chair and CIHR staff to establish a review sequence or a meeting agenda, and to identify the eligible applications for streamlining; and,
  • Work with the Chair and CIHR staff to manage conflicts of interest of committee members.

During the committee meeting, the Scientific Officers will:

  • Take official notes of the committee discussions for each application (SO Notes); these will be sent to the applicants along with the reviewers’ reports;
  • The SO Notes should provide the applicants with insight into the committee discussion of their applications;
  • They should be clear and concise, and give objective and constructive feedback to the applicants. They should:
    • Include the strengths and weaknesses of the applications discussed by the committee;
    • Address the issues that had the greatest impact on the evaluation, as they relate to the program’s evaluation criteria;
    • Address aspects of the committee discussions that were not captured in the reviewers’ reports;
    • Describe how reviewer disagreements, as seen in the individual reviewer reports, were reconciled by identifying which view was favored by the committee.
  • Read back the SO Notes to the committee for validation and for additional input before a consensus rating on each application is reached by the assigned reviewers, and all the members’ votes are cast;
  • Read back the summary of recommendations to the budget and term of the project, and ensure that special considerations related to ethics and/or other issues are also recorded in the SO Notes, if applicable; and,
  • Not rate applications or vote during the committee meeting.

Note

CIHR will not edit or modify the SO Notes without the Scientific Officers’ approval

At the end of the committee meeting, the Scientific Officers will:

  • Finalize and post the SO Notes directly on ResearchNet or send them to the CIHR staff for posting no later than 5 business days after the committee meeting.

5.3.3 Reviewers

Reviewers are committee members assigned to review one or more applications for which they have declared expertise and are not in conflict with. Applications are assigned to three reviewers who participate in their rating and submit in-depth written reports that are provided to the applicants. They present their reviews at the committee meeting where they lead the discussion, and vote during the meeting. They also participate in the discussion and rating of other applications before the committee for which they are not in conflict.

Before the committee meeting, reviewers must:

  • Evaluate each of the applications assigned to them by providing a critical assessment of the applications, as well as providing constructive feedback based on the program’s objectives and adjudication criteria described in the funding opportunity by:
    • Considering all factors and the strengths and weaknesses of the applications in relation to each adjudication criterion;
    • Focusing their comments on the factors most relevant to their ratings;
    • Providing feedback that is original and written in their own words;
    • Providing comments on the integration of sex (as a biological variable) and/or gender (as a socio-cultural determinant of health) in the applications, if applicable;
    • Providing comments on the budgets requested with a formal recommendation to CIHR in the “Budget” section, including clear and detailed reasons for any recommended budget or term cuts, if applicable;
    • Providing comments on issues that they feel should be flagged, as required. These concerns should not influence the rating or budget recommendations, unless they bear on the scientific merit of the applications.
  • Provide their initial ratings to one decimal place on ResearchNet;
  • Divide their applications into a top (competitive) and bottom (non-competitive) group, based on their overall quality; and
  • Familiarize themselves with the applications to be assessed by their committee as this will facilitate discussions at the face-to-face committee meeting.

During the committee meeting, reviewers will:

  • Present to the committee the review of their assigned applications (see section 4.2.2 above, for the adjudication criteria);
  • Participate in the committee discussions; and,
  • Vote on all the applications discussed by the committee and for which they are not in conflict with

At the end of the committee meeting, reviewers will:

  • Have the option to revise their reviews within 5 business days after the end of the committee meeting.
Date modified: