Mock Review Toolkit: Selecting a Simulation
On this page
2. Selecting a Simulation
Long description
Tasks:
- Questionnaire – Goals, Audience, and Scope of Simulation
- Select a Simulation Type
Week 1: Light Simulation
Week 1: Full Simulation
Week 1: Internal Simulation
Use the questionnaire below to define the goals, audience and scope of your simulation and, based on your answers, use the proceeding descriptions of each simulation type to select a simulation.
2.1 Questionnaire – Goals, Audience, and Scope of Simulation
What are your Goals?
- A quick overview of peer review, including a short mock Committee meeting as part of a workshop? If so, consider the Light Simulation.
- To fully simulate the Project Grant Peer Review process, including a fully simulated committee meeting? If so, consider the Full Simulation.
- To conduct an internal review process for applications from your own institution to improve quality? If so, the Internal Simulation is appropriate.
Who is your Audience and what is their Scope of Expertise?
- Are you planning for participants with a broad scope of expertise and would like to have Committees focused on a broad Theme.
If so, consider the Light Simulation. - Are you planning for participants with varying expertise and require flexibility in the form of multiple Committees focused on different Themes, Areas of Science or Project Mandates?
If so, consider the Full Simulation.
How many mock applications can you dedicate to the committee meeting?
- Do you have fewer mock applications on hand, or only a limited time set aside for the Committee meeting itself?
If so, the Light Simulation is more appropriate - Are you planning to have many mock applications, and are looking to dedicate a half-day to a simulated Committee meeting?
If so, the Full or Internal Simulation is more appropriate
What is your overall timeline?
- Do you have to run the simulation on a compressed timeline, or on short notice?
If so, the Light Simulation might be preferable
2.2 Select a Simulation Type
Light Simulation
Goal
This simulation is designed to provide a first glimpse of the peer review process at CIHR.
Audience
Can be used by a broad audience consisting of a broad scope of expertise and can accommodate the greatest number of participants.
Description
Reviewers are assigned 1-2 application(s) to review at home prior to attending the mock review meeting. At the meeting, participants will breakout into smaller Committees of 6-8 Reviewers with an experienced reviewer acting as Chair for each. Reviewers will discuss the mock applications in their respective Committees before reconvening as a larger group for a facilitated discussion focused on peer review.
Applications
Mock applications are provided through the Toolkit or provided by the Facilitator. Suggest 1-2 applications per simulation, where all participants across Committees review the same applications covering a broad Theme.
Please note, these are simply suggestions for the number of committees and their scope. You may choose to alter the parameters of a simulation type as needed. For example, in the Light Simulation, each committee can instead focus on a specific Area of Science or Project Mandate where each committee reviews a unique set of applications.
Logistics
Time Required
- Total Length: 4-8 weeks
- Planning: ~4 weeks
- Reviewer preparation: ~1-2 hours (~3-4 with optional Pre-simulation Training Session)
- Committee meeting: 1.5-3 hours *based on 1-3 applications total for review
Participants
- 2 Facilitators
- 6-8 Reviewers per Committee (ideal)
- # of Reviewers and Committees depends on available Chairs
- 1 Chair per Committee
- 1 or more Scientific Officers per Committee
- Reviewers or Facilitators may rotate the role of a Scientific Officer.
Full Simulation
Goal
This simulation is designed to replicate the Project Grant peer review process at CIHR.
Audience
Suited to trainees, postdoctoral fellows and early career researchers who would like to get hands-on peer review experience. Participant scope of expertise may be diverse.
Description
Reviewers are assigned 2-4 applications to review prior to attending the Committee meeting. Participants will convene as a Committee, where they will present, discuss, and score their assigned applications, with an experienced Reviewer acting as Chair.
Applications
Mock applications are provided through the Toolkit or provided by the Facilitator. Suggest 4-8 applications per Committee. Each application is reviewed by up to 3 Reviewers and each Reviewer reviews 2-4 applications. Application pool can be diverse across Committees covering various Themes, Areas of Science or Project Committee Mandates.
Please note, these are simply suggestions for the number of Committees and their scope. You may choose to alter the parameters of a simulation type as needed. For example, in the Full Simulation, a single Committee can be run with a focus on a broad Theme instead.
Logistics
Time Required
- Total Length: 9-11 weeks
- Planning: ~10 weeks
- Reviewer preparation: ~2-3 hours (4-5 hours with optional Pre-simulation Training Session)
- Committee meeting: 3-5 hours
Participants
- 2 Facilitators
- 9 Reviewers per Committee (ideal)
- 1 Chair per Committee
- 1 or more Scientific Officers per Committee
- Reviewers or Facilitators may rotate the role of a Scientific Officer to gain experience.
Internal Simulation
Goal
This simulation is designed to be used as an internal review process. The Facilitator would seek draft applications for an upcoming Project Grant competition from their institution for use in the simulation.
Audience
This simulation is applicable to both less experienced and highly experienced Reviewers. Participant expertise can be broad (all Themes) or narrow (a particular Area of Science)
Description
Follows same process as Full Simulation; however, both the Scientific Officer and Reviewer written comments will be provided to the applicant following the Committee meeting to improve their grant prior to submission.
Applications
Internal applications provided by own institution. Suggest 4-8 applications per Committee. Each application is reviewed by up to 3 Reviewers. Application pool can be broad (cover all Themes) or narrow (a specific Theme, Area of Science or Project Committee Mandate) depending on your planned Audience.
Logistics
Time Required
- Total Length: 9-11 weeks
- Must begin a minimum of 16 weeks prior to the Project Grant deadline
- Planning: ~12-14 weeks
- Reviewer preparation: ~3-4 hours (5-6 with optional Pre-simulation Training Session)
- Committee meeting: 3-5 hours
Participants
- 2 Facilitators
- 9 Reviewers per Committee (ideal)
- 1 Chair per Committee
- 1 or more Scientific Officers per Committee
- Reviewers or Facilitators may rotate the role of a Scientific Officer to gain experience.
- Date modified: