Peer Review Guide
Pilot of the Tri-Agency Interdisciplinary Peer Review Committee

1. Introduction

The aim of this document is to serve as a resource for reviewers adjudicating applications submitted to the Tri-Agency Interdisciplinary Peer Review Committee. This shared peer review committee is accessed via the following Gateway Funding Opportunities: SSHRC Insight Grants and Insight Development Grants, CIHR Fall and Spring Project Grant, and NSERC Discovery Horizons.

On behalf of SSHRC, CIHR, and NSERC, we would like to thank the peer review committee members. These dedicated experts who generously give their time and expertise are essential to ensuring the success of the peer review process.

If you need further guidance or would like to have more information about this review process, please do not hesitate to contact:

2. Objective

Individuals applying for SSHRC Insight Grants and Insight Development Grants, CIHR Fall and Spring Project Grant, and NSERC Discovery Horizons will be eligible to be considered for evaluation by to the Tri-Agency Interdisciplinary Peer Review Committee.  This committee provides an option for researchers working in interdisciplinary research to direct their application to a committee with expertise from across the social sciences, humanities, natural sciences, engineering, and health sciences.

The agencies recognize that the terms "multidisciplinary," "transdisciplinary," and "interdisciplinary" have different connotations and that there may be a lack of consistency in their usage across and between research disciplines. For the purpose of this committee, "interdisciplinary" is used as an umbrella term to refer to all three concepts inclusively.

3. Review Process

3.1 Important Dates

Event Timeline
NSERC Discovery Horizons Letter of Intent deadline June
CIHR Fall Project Grant Registration deadline August
CIHR Fall Project Grant Application deadline September
SSHRC Insight Grant Application deadline October
NSERC Discovery Horizons Application deadline October
Preliminary TAIPR Committee Orientation meeting(s) Late Fall
CIHR Fall Project Grant committee meeting December
NSERC Discovery Horizons committee meeting January
SSHRC Insight Development Grants Application deadline February
SSHRC Insight Grants committee meeting Late February
CIHR Spring Project Grant registration deadline February
CIHR Spring Project Grant application deadline March
SSHRC Insight Development Grant committee meeting Early May
CIHR Spring Project Grant committee meeting June
Agency Funding Decisions and Anticipated Notification of decision to applicants Based on Funding Opportunity Timelines

In addition to the above timelines, the following activities will take place for each Gateway Funding Opportunity according to its timelines:

Conflict of Interest and Ability to Review deadline Approximately 1 month after each full application deadline.
Release of Committee Assignments Approximately 4-6 weeks before each committee meeting.
Preliminary scores and written report due date 1 week before the committee meeting
Final written report due date 1 week after the committee meeting

3.2 Assigning Applications

Following the submission of full applications, committee members will be asked to complete the Ability to Review exercise. To do so, members are given access to the summaries of proposed research projects to declare any conflicts of interest and indicate their level of expertise (i.e., High/Medium/Low/Not enough expertise). Given the interdisciplinarity nature of the committee, members are encouraged to take a broad view of their own expertise as it relates to interdisciplinary applications and provide context regarding their expertise, where relevant. The workload of the reviewers will be balanced across the applications from the three agencies.

Each application will be assigned to 5 committee members: 3 as Reviewers and 2 as Readers (see section 4, below, for more details on the roles):

3.2.1 Committee Orientation and Calibration

Agency staff will convene preliminary meetings of the Tri-Agency Interdisciplinary Peer Review Committee in advance of the peer review meeting for the purposes of committee orientation and committee calibration.

As this committee’s processes and evaluation criteria have been harmonized and will include members from across the social sciences, humanities, natural sciences, engineering, and health sciences, committee orientation provides an opportunity to explain processes and procedures that will apply to this committee, as well as address any questions from reviewers stemming from their previous peer review experiences. Agency staff will also introduce the peer review platform and other related information technologies being used to support the peer review meeting.

The committee calibration offers an opportunity for discussion of the consistent use of evaluation criteria and the scoring system, time-management strategies for the adjudication meeting, and other topics.

3.3 Review of Applications before the peer review meeting

All applications must be treated as protected information and all committee members will sign a tri-agency Conflict of Interest and Confidentiality Agreement.  Committee members must not divulge application contents to others or use any information herein for any other purpose than peer review. Committee members are also asked to ensure that all Peer Review materials are handled safely and disposed of according to the document “Guide on Handling Documents used in Peer Review”.

3.4 Review Process

Reviewers will be asked to submit a written review whereby they will provide strengths and weaknesses of the project in relation to the Tri-Agency Interdisciplinary Peer Review Committee evaluation criteria for the competition. Reviewers and Readers should consider, but are not bound by, the External Assessment Reports.

Note: A template will be provided to the reviewers to facilitate their written report (which can then be copied and pasted into the online system).

After the reviewers have submitted their final reviews, they will be able to access the other finalized reviewers’ reports for that application. These reports can be used in preparation for the discussion at the committee meeting.

If applicable, reviewers may also comment on the budget requested and make a formal recommendation, including clear and detailed reasons for any recommended budget cuts.

3.4.1 Evaluation Criteria and rating scale

To support the strategic objectives of this funding opportunity, the following evaluation criteria will be used. Reviewers will enter scores for each of the three criteria, which will be given equal weight.

Criteria Elements
Merit of the Proposed Research Quality of the Proposal The clarity, scope and originality of research questions and objectives
Methods and Approaches

Appropriateness of the proposed methods and approaches, including the integration of interdisciplinary approaches to achieve the project goals.

Appropriate incorporation of sex, gender and diversity in the research design, if applicable

Evidence of commitment to the development of active and meaningful partnerships with appropriate individuals and organizations, if applicable

Feasibility

Extent to which the budget is appropriate and justified in relation to the proposed activities

Appropriateness of the environment (academic institution and/or other organization) to enable the conduct and success of the proposed activities

Anticipated Outcomes Impact Potential for the outcomes of the proposed activities to have influence and impact
Knowledge mobilization and dissemination Quality and appropriateness of knowledge mobilization plans, including effective dissemination, exchange and engagement with stakeholders within and/or beyond the research community, where applicable
Training Plan

Extent to which the proposed activities incorporate plans for the training and development of highly qualified personnel

Appropriate inclusion of considerations of equity, diversity and inclusion in the training plan

Applicant(s) Record Expertise and Experience

Appropriate expertise to undertake the proposed research, including complementary and interdisciplinary knowledge, expertise and experience

Experience and track record of the investigator(s) – importance, originality, quality and impact of past research

Past Contribution to Training of Highly Qualified Personnel (HQP) Quality and extent of past contributions to the training of HQP

See section 5 below for a more detailed discussion of the evaluation criteria.

The following rating scale will be used:

Descriptor Range Outcome Definition
Outstanding 4.5 – 4.9 Fundable range The application excels in most or all relevant aspects. Any shortcomings are minimal.
Excellent 4.0 – 4.4 The application excels in many relevant aspects, and reasonably addresses all others. Certain improvements are possible.
Very good 3.5 – 3.9 The application excels in some relevant aspects, and reasonably addresses all others. Some improvements are necessary.
Fair 3.0 – 3.4 Non-fundable range The application broadly addresses relevant aspects. Major revisions are required.
Poor 0.0 – 2.9 The application fails to provide convincing information and/or has serious inherent flaws or gaps.

Applications are submitted according to the guidelines, formatting, page limits, required attachments etc. of the relevant funding opportunity. Applicants should not be penalized as long as they are following the rules appropriate to their individual agencies. To facilitate this, applications will be reviewed in bundles.

3.4.2 Peer Review Meetings

The Tri-Agency Interdisciplinary Peer Review Committee will meet over multiple sessions to review the eligible and relevant applications submitted via the agencies’ funding opportunities. As noted above, the applications will be reviewed in bundles by funding opportunity and the review may occur over a number of sessions lasting one or more days, taking place in accordance with the funding opportunity timelines. Please note that the agencies may streamline applications based on existing practices, and as a result the committee may not discuss all files.

One reviewer will be assigned as ‘primary’ reviewer and will open the discussion by presenting their assessment of the strengths and weaknesses of the proposal according to their perspective and specific to each of the evaluation criteria for the funding opportunity.  The remaining reviewers and readers follow and supplement the discussion by concentrating on points of agreement or disagreement, and elaborating points not yet addressed, with attention given to their specific perspective. The open discussion will be structured and discussion will proceed according to the order of the evaluation criteria.

Once the discussion is complete, the Chair will invite reviewers and readers to confidentially vote on the application.

Following the voting, the panel will review the requested budget and make a formal recommendation as necessary. Any recommended budget modifications, if applicable, must be detailed and clearly outlined. Any other concerns should be discussed and, if necessary, flagged for staff to address (i.e., ethics, eligibility, human stem cells, etc.). These issues are not to be considered as criteria for evaluation, except if they impact the quality of the application. In such cases, reviewers should bring these issues to the committee coordinator as soon as possible in order to seek guidance on how to address these in their review and committee discussion.

Upon conclusion of the review and rating of all applications, reviewers will be asked to provide feedback to the funding agencies regarding the pilot and the effectiveness and functioning of the committee and the peer review process in its entirety.

3.5 After the peer review meeting

Reviewers who wish to make changes to their reviews will have one week to do so after the meeting. Reviewer reports may be made available (without the reviewer’s name or score) to the applicants as part of the feedback package provided by their agency. As such, reviewers should refrain from divulging their identities in their reviews to ensure the confidentiality of the review process.

The review should be clear and concise, using objective and non-inflammatory language, and include justification (see, for example, the CIHR College of Reviewer’s Review Quality webpage). Constructive advice to the applicant will allow them to improve the quality and efficiency of the proposed research. Note: in circumstances where the Chair or staff deems that the review expresses comments that could be construed as sarcastic, flippant, arrogant, or inappropriate in any way or contain factually incorrect information regarding funding agency’s policies, the funding agencies reserve the right to remove such information from the review.

While applications from the three funding opportunities will be reviewed by the shared Tri-Agency Interdisciplinary Peer Review Committee, funding decisions will be made, funding will be provided, and grants will be administered by the agency to which the application was submitted, in accordance with that agency’s respective guidelines and decision making processes. Similarly, applicant feedback packages will follow the format of the agency to which the application was submitted.

4. Roles and Responsibilities

The Tri-Agency Interdisciplinary Peer Review Committee consists of a Chair, three Scientific Officers, and Peer Review Committee Members. In addition to the committee, External Assessors will also be engaged to support the committee through the provision of External Assessor Reports. The committee is supported by program staff at each of the three agencies.

4.1 Chair

The committee Chair is responsible for ensuring that the peer review committee functions smoothly, effectively and objectively. They establish a positive, constructive, and fair-minded environment in which the applications are to be evaluated. The Chair (along with the Scientific Officers and program staff) fulfills an oversight role and does not rate applications. Their responsibilities include:

4.2 Scientific Officers (SO)

The Scientific Officers (SO) are responsible for supporting the Chair during the peer review process as appropriate, including by acting as back-up chairs when the Chair has a conflict of interest. In addition, the SO will take notes on the committee proceedings which may form part of the applicant feedback packages. Each agency recruits an SO. In addition to the duties shared with the Chair, the Scientific Officer:

4.3 Peer Review Committee Members

Peer Review Committee Members are recruited from across the social sciences, humanities, natural sciences, engineering, and health sciences. They will be assigned as Reviewers and Readers to applications assessed by the committee, as supported by External Assessor reports. The primary responsibilities of the peer review committee members are to:

4.4 External Assessors

The funding agencies will seek to obtain, but cannot guarantee, at least two External Assessment Reports from subject matter experts for each application. Committee members should take into consideration, but are not bound by, these reports while conducting their reviews. External Assessors are not part of the committee. The role of the External Assessors is to provide written expert reviews of the application(s) assigned to them.  External Assessors will also abide by the agencies’ guidelines on confidentiality and conflict of interest.

4.5 Funding Agency Staff

Funding agency staff are involved throughout the funding opportunity process, including supporting the Peer Review Committee in its work. The responsibilities of Funding Agency staff include:

5. Description of Evaluation Criteria

The Tri-Agency Interdisciplinary Peer Review Committee employs harmonized evaluation criteria. While the specific definition of terms may vary between agency-specific funding opportunities, applicants and committee members will consider the harmonized description of the evaluation criteria. Please note that not every participating funding opportunity has the same requirements in their applications and that different amounts of space may be provided between applications. Applicants and committee members are asked to communicate with their agency contacts if they have any questions.

Merit of the Proposed Research

Quality of the Proposal

Methods and Approaches

Feasibility

Anticipated Outcomes

Impact

Knowledge mobilization and dissemination

Training Plan

Important: trainee demographic data is not requested, nor required to assess impacts related to equity, diversity and inclusion in the research and training environment.

Applicant(s) Record

Expertise and Experience

Past Contribution to Training of Highly Qualified Personnel (HQP)

6. Resources

Applicants should review the funding opportunity specific guidelines and documents available on their websites:

In addition to the funding opportunity specific resources, applicants should also review the following tri-agency guidelines:

Date modified: