Mock Review Toolkit: Running the Simulation

If the Committee meeting is to be in-person, we suggest the table be setup as depicted below to better facilitate discussion.

The Facilitator is responsible for organizing the Committee meeting(s) and ensuring that everyone has the appropriate documentation.

The Facilitator will:

• provide all necessary documentation
• discuss conflicts of interest
• clarify policies and/or administrative processes
• keep track of the time
• ensure Chair knows when it is time to take a break

It is the Chair's responsibility to ensure that the review committee functions smoothly, effectively and objectively, and that a positive, constructive, fair-minded environment in which research proposals are evaluated is established and maintained.

The Chair will:

• provide opening remarks to the committee, including an outline of the structure and agenda of the day.
• explain the meeting process to the committee, including the review of SGBA considerations in the applications.
• briefly discuss the budget and term components and remind Reviewers that they are not part of the Committee meeting.
• ensure that all committee members who are in conflict with an application leave the meeting room before the discussion of the application.
• appoint a delegate as Chair or Scientific Officer when either individual leaves the meeting room due to a conflict of interest with an application or for any other reasons.
• fulfill an oversight role – does not rate applications nor vote during the Committee meeting.
• ensure the involvement of the entire committee in evaluating each application.
• work with the Scientific Officers, as required, to summarize the discussion around each application, before the consensus rating is reached.
• ensure that a consensus rating is reached by the assigned Reviewers.
• ensure that specific ethical concerns and other CIHR requirements are addressed, and that any related discussion is captured in the Scientific Officer notes.

Reviewers can rotate the role of an SO to gain experience. If this is the case, ensure that all “SO Reviewers” receive a copy of the SO Note Template (available in Preparation and Room Setup above). You can also include a Mock SO Note (included with some, but not all Mock Applications) to provide Reviewers with an example.

The Scientific Officer (SO) assumes the role of note taker. The SO will:

• take official notes of the committee discussions for each application (SO Notes). The SO Notes should provide the applicants with insight into the committee discussion of their applications. They should be clear and concise and give objective and constructive feedback to the applicants. They should:
  • include the strengths and weaknesses of the applications discussed by the committee.
  • address the issues that had the greatest impact on the evaluation, as they relate to the program's evaluation criteria.
  • address aspects of the committee discussions that were not captured in the Reviewers' reports.
  • describe how Reviewer disagreements, as seen in the individual Reviewer reports, were reconciled by identifying which view was favored by the committee.
• read back the SO Notes to the committee for validation and for additional input before a consensus rating on each application is reached by the assigned Reviewers, and all the members' votes are cast.
• ensure that special considerations related to ethics and/or other issues are also recorded in the SO Notes, if applicable.

The Reviewer evaluates each of the applications assigned to them by providing a critical assessment of the applications, as well as constructive feedback based on the program's objectives and adjudication criteria described in the funding opportunity.

The Reviewer will:

• consider all factors and the strengths and weaknesses of the applications in relation to each adjudication criterion.
• focus their comments on the factors most relevant to their ratings.
• provide comments on the integration of sex (as a biological variable) and/or gender (as a socio-cultural determinant of health) in the applications, if applicable.
• provide comments on the budgets requested and a formal recommendation to CIHR in the "Budget" section, including clear and detailed reasons for any recommended budget or term cuts, if applicable (*Note: discussion of budget and term support is not currently within scope of this Mock Review Toolkit).
• provide comments on issues that they feel should be flagged, as required. These concerns should not influence the rating or budget recommendations, unless they bear on the scientific merit of the applications.
• provide their initial ratings to one decimal place using the provided Mock Review Templates
• familiarize themselves in advance of the Committee meeting with the applications to be assessed by their committee as this will facilitate discussions at the face-to-face Committee meeting.
• present to the committee the review of their assigned applications.
• participate in the committee discussions.
• vote on all the applications discussed by the committee and for which they are not in conflict with.

Reviewer Scripts

Primary Reviewer(s) Script:

• Once Chair introduces Reviewers, present initial scores to committee.
• Present a brief overview of the application, premise of the research, study design etc.
• Discuss strengths and weaknesses.
• Listen to other reviews.
• Following Secondary Reviewer(s) discuss consensus score.
• Discuss further if necessary.
• Confirm content of notes from Scientific Officer at the end of the discussion.

Secondary Reviewer(s) Script:

• Once Chair introduces Reviewers, present initial scores to committee.
• Listen to Primary Reviewer(s) overview of the application and comments.
• Present strengths and weaknesses not mentioned by previous Reviewers.
• Once all discussion has taken place, Reviewers discuss consensus score.
• Discuss further if necessary.
• Confirm content of notes from Scientific officer at the end of the discussion.