HSIF Evaluation Criteria / Notes for Reviewer

Note

These notes should help reviewers navigate the FO during peer review. However, an application should be evaluated in its entirety.

Peer review will be conducted in accordance with the CIHR's peer review policies and procedures and in accordance with the Declaration on Research Assessment (DORA). Reviewers are encouraged to consider a range of contributions (e.g., research publications, reports, books, guidelines, policy reports, white papers, datasets, code, tools, training and mentorship, volunteerism, community engagement, civic engagement, public/community engagement, partnerships and collaborations) and impacts (e.g., influence on policy and practice, health outcomes, societal outcomes, and distinctions-based, meaningful and culturally safe health research). Reviewers should also consider, if disclosed, the applicant's context (e.g., leave history, career stage, area(s) of research, experiential knowledge, diverse career paths, family responsibilities, pandemic impact) and how it may have impacted the applicant's progress. Additionally, reviewers are directed to avoid using publication metrics in isolation (e.g., number of publications and citations, size/number of research grants), and to avoid using journal-based metrics (e.g., Journal Impact Factors; H-index). Reviewers are expected to familiarize themseles with the following material: DORA at CIHR, and CIHR's DORA-related resources for peer reviewers. Reviewers are also invited to refer to the DORA Frequently Asked Questions and the College of Reviewers DORA theme Skill Builders.

Reviewers should take into account the career stage of the applicants to better asess and calibrate their set of applications (e.g., direct entry to fellowship from PhD vs entry after several years outside academia).

1. Achievement and potential of the Nominated Principal Applicant and Relevance to HSI Program Objectives (25%)

Evaluation Criteria Information Source Notes for Reviewers
a. Professional achievements and leadership potential (e.g., community engagement, civic engagement, volunteerism, involvement in work/academic life, other), given training/career stage and leave context
  • Applicant's description of "relevant accomplishments and abilities"
  • Applicant's TCV
  • Review the identified information sources for examples of professional recognitions, voluntary contributions, civic engagement, professional development training, relevant work experience, etc.
  • Examine the characteristics and skills of the applicant (leadership, project management/organizational skills, communication, knowledge translation, critical thinking, interest in evidence-informed health system improvement)
  • Assess relative to your expectations of someone with their professional experience. Keep in mind that not all applicants would have the same opportunity to pursue volunteer opportunities.
b. Academic achievements and activities (e.g., policy reports, white papers, research publications, books, guidelines, datasets, tools, training and mentorship, partnerships and collaborations, media, public/community engagement, other) given career stage and leave context
  • Applicant's description of "relevant accomplishments and abilities"
  • Applicant's TCV
  • Academic transcripts (for doctoral applicants only)
  • Review the identified information sources for evidence of awards, scholarships, publications and presentations, knowledge translation activities, relevant course work and academic training, and course work performance (types of courses, course load, grades, overall average, trend [give credit for steadily improving or consistently good performance])
  • Assess relative to your expectations of someone with their academic experience.
  • Assess the quality and impact of contributions directly. Avoid using publication metrics in isolation as well as journal-based metrics as sole indicators of the quality and impact of a researcher’s overall productivity/track record or the quality and impact of an individual research publication.
c. For applications involving Indigenous Health Research: demonstration of the capacity and track record to work and engage with Indigenous communities in a meaningful and culturally safe way.
  • Applicant's description of "relevant accomplishments and abilities"
  • Applicant's TCV
  • Applicant's "Indigenous Health Research Cultural Safety Form" attachment

Consider:

  • The applicant's experience and ability to work and engage with Indigenous communities in a meaningful and culturally safe way

2. Quality of the host partner and organization's and academic institution's training environment, supervision and mentorship plan (35%)

Evaluation Criteria Information Source Notes for Reviewers
a. Quality of supervisors’ training, supervision, and mentorship plan and its alignment to the applicant’s fellowship objectives and professional development goals, including demonstration of commitment to: meaningfully embed and integrate the applicant within the organization; co-train and co-mentor the applicant in a dual mentorship model involving both supervisors working together in support of the applicant; and promote the applicant’s success and impact in the award, their professional development and career advancement, and the demonstration of the mentors’ commitment to supporting a longer-term career pathway.
  • Applicant's description of "Training environment, supervision and mentorship plan"
  • Supervisors' letters of support
  • "Team Expertise and Experience" document

Consider:

  • Evidence of quality mentorship from a health system leader in the host partner organization and an academic supervisor
  • Evidence of a team-based, collaborative approach to supervision and mentorship
  • Type and breadth of opportunities for professional development (e.g., regular opportunities to meet with supervisors, professional development training courses, involvement in senior-level meetings, opportunities to meet executive leaders within the organization and relevant partner organizations)
  • Alignment of plan with CHSPRA's enriched core competencies and with applicant's three selected competencies for development
  • Extent to which plan will help the applicant achieve success and impact in the award and accelerate their professional growth, career advancement, and longer-term career aspirations
b. Demonstration of supervisors' (health system and academic) commitment to the program objectives: track record of expertise of the academic supervisor in health services and policy research or related fields (e.g., health systems, population and public health, data science, epidemiology, health policy, service delivery, etc.); track record and expertise of the health system supervisor in leading and/or influencing evidence-informed health policy/system/practice improvement; and capacity within their organizational roles to champion embedded research, support the applicant’s embedded research and act upon it (i.e., implement) to inform program/policy/practice decision-making.
  • "Team Experience and Expertise" document
  • Applicant's description of "Training environment, supervision and mentorship plan"
  • Supervisors' letters of support

Consider:

  • Evidence of commitment to evidence-informed health system improvement and/or learning health systems
  • Evidence of health system supervisor's experience/expertise in leading and/or influencing evidence-informed health system improvement
  • Evidence of the academic supervisor's experience/expertise in HSPR related fields
  • Evidence, by way o clearly articulated and thoughtful strategies for achievement, that the host partner organization is committed to the HSI program objective of harnessing research-trained individuals for improved decision-making
  • Experience mentoring and supervising research trainees and/or employees
  • Commitment to mentoring the applicant and accelerating their professional growth
  • Evidence that academic supervisor is supportive of the experiential learning opportunity

Bear in mind the Primary Supervisor from the health system organization may not have experience supervising trainees but may have relevant experience supervising and mentoring employees
c. Quality of the learning environment (host partner organization and university), potential for the learning environment to expose the applicant to health system decision-making and the professional skills to influence evidence-informed decision-making, and suitability of the learning environments given applicant's motivation for applying and targeted enriched core competencies.
  • Applicant's description of "Training environment, supervision and mentorship plan"
  • Supervisors' letters of support
  • Applicant's "motivation for applying and value-add" statement
  • "Team Expertise and Experience" document

Consider:

  • Whether the learning environment will expose the applicant to health system decision-making and provide the opportunity for the applicant to develop and apply their professional skills to contribute to evidence-informed decision-making
  • Whether the learning environment is equitable and inclusive as well as one in which an applicant will be inspired and challenged
  • Whether adequate supports and resources are available to the applicant in support of their pursuit of the organization's impact goal (e.g., access to data, office space, introductions to key partners and stakeholders, etc.)
  • Extent to which the applicant will be meaningfully integrated into the organization (e.g., involvement in team meetings, and external meetings with relevant partners, opportunities to be involved in other projects within the organization)
  • Evidence of joint commitment from both supervisors to work together to help the applicant succeed
  • Fit of learning environment given applicant's motivations for applying and targeted enriched competencies for professional growth

Bear in mind the host partner organization has, by the review stage, been deemed eligible and that this program encourages the involvement of private for-profit, not-for-profit and public sector health system and related organizations at federal, provincial, regional and local levels
d. Demonstration of the supervisors' commitment to Equity, Diversity and Inclusion in the training environments, professional development training activities, mentorship and/or other activities, as applicable. Additional guidance can be found on the Best Practices in Equity, Diversity and inclusion in Research webpage and in the NSERC - NSERC guide on integrating equity, diversity and inclusion considerations in research
  • Applicant's description of "Training environment, supervision and mentorship plan"
  • Supervisors' letters of support

Consider:

  • Whether Equity, Diversity and Inclusion considerations have been incorporated within the training environments (academic and health system), professional development training activities, mentorship plan(s) and/or other activities, as applicable, and the quality and appropriateness of those considerations

e. For applications involving First Nations, Inuit and/or Métis Peoples:

(i) extent to which the overall research team has the necessary knowledge, expertise and experience in Indigenous health research, and complementarity of expertise and synergistic potential to conduct Indigenous health research that is scientifically excellent, community-driven and culturally safe;

(ii) conduciveness of the training environment to supporting the rights of Indigenous Peoples and Indigenous self-determination.
  • Applicant's description of "Training environment, supervision and mentorship plan"
  • Supervisors' letters of support
  • "Team Experience and Expertise" document

Consider:

  • Whether the supervisors have a demonstrated capacity and track record of working and engaging with Indigenous communities in a meaningful and culturally safe way
  • The conduciveness of the training environment to supporting the rights of Indigenous Peoples and Indigenous self-determination
f. For applications involving patient-oriented research and/or engagement with equity-denied communities (e.g., women, gender-diverse people, communities impacted by racism, persons with disabilities, Indigenous Peoples, members of 2SLGBTQI+ communities): evidence of meaningful, safe and appropriate engagement experience with these communities.
  • Applicant's description of "Training environment, supervision and mentorship plan"
  • Applicant's TCV
  • "Team Experience and Expertise" document
  • Applicant's "EDI and Community Engagement Plan"

Consider (where applicable):

  • Evidence of the supervisors' experience with meaningful, safe and appropriate engagement with equity-denied communities

3. Quality, potential impact and feasibility of the applicant's project/program of work proposal (20%)

Evaluation Criteria Information Source Notes for Reviewers
a. Clarity of the rationale for how the project/program of work proposal addresses a critical challenge/impact goal faced by the host partner organization (i.e., importance and relevance) and the impact potential within the organization.
  • Applicant's description of "Project/Program of work proposal"
  • Primary Supervisor's letter of support

Consider:

  • The clarity and importance of the organization's impact goal
  • The extent to which the proposed project/program of work will address the goal
  • The potential for impact that the project/program will have on enhancing the organization's achievement of the goal (relevance and impact potential)
  • The challenge and excitement of the project in which the applicant will be involved
b. Quality of the embedded research program of work approach, relevant theories and methods, appropriateness of the defined outcomes in relation to the organization’s impact goal, and feasibility of the embedded program of work proposal (i.e., appropriateness of the scope of work in relation to the duration of the award and strength of analysis of potential risks and challenges to implementing the work).
  • Applicant's description of "Project/Program of work proposal"
  • Applicant's "Work plan table"

Consider:

  • The appropriateness of the proposed project design and approach, including theories, methods and outcomes in relation to the organization's impact goal
  • The feasibility of accomplishing the project within the time requirements
  • The appropriateness of strategies to mitigate anticipated risks/challenges in accomplishing the project/program of work in the award's timeframe
c. Quality of the knowledge mobilization plan (including co-design, knowledge user engagement, collaborations/partnerships across relevant academic-system-community-other partners, evidence support strategies, data/evidence input and feedback mechanisms).
  • Applicant's description of "Project/Program of work proposal"
  • Applicant's "Work plan table"

Consider:

  • The appropriateness and quality of the knowledge mobilization plan
d. Quality and appropriateness of the proposed plan for how equity, diversity and inclusion (EDI) will be integrated into the research design (refer to Best Practices in Equity, Diversity and Inclusion in Research). This includes consideration of biological variables (e.g., sex, age) and/or the impacts of inequalities and discrimination based on socio-economic status, sexual orientation, gender, race, culture, age, language, Indigeneity and/or disability, as applicable.
  • Applicant's description of "Project/Program of work proposal"
  • Applicant's "EDI and Community Engagement Plan"

Consider:

  • Extent to which EDI is integrated within research project/program and knowledge mobilization plans and appropriateness and meaningfulness of the approach(es) for doing so.

e. For any applications involving First Nations, Inuit and/or Métis Peoples:

(i) Quality and appropriateness of the description of the meaningful and culturally safe engagement plan with Indigenous Peoples that abide to TCPS 2 (2022): Chapter 9 - Research Involving the First Nations, Inuit and Métis Peoples of Canada.

(ii) Extent to which the proposed project recognizes the rights of Indigenous Peoples and the Indigenous self-determination and self-governance of Indigenous Peoples, including Indigenous data, are respected and integrated, such as following the First Nations Principles of OCAP® Footnote 1(Ownership, Control, Access and Possession), the CARE Principles for Indigenous Data Governance, and/or FAIR Principles.
  • Applicant's description of "Project/Program of work proposal"
  • Applicant's Indigenous Health Research Cultural Safety Form

Consider:

  • Whether the approaches and methods of the proposed research project respect Indigenous rights, values and ways of knowing and sharing
  • Whether the proposed research project has the potential to produce valued outcomes from the perspective of First Nations, Inuit and/or Métis participants and Indigenous Peoples more broadly
f. For applications involving historically equity-denied communities: quality and appropriateness of the approach and engagement plan to engage/integrate community representation throughout the research with ethical, safe, and methodologically sound engagement strategies, and demonstration that the team will consider equity in research to mitigate the power imbalance between the researcher and community members.
  • Applicant's description of "Project/Program of work proposal"
  • Applicant's "EDI and Community Engagement Plan"

Consider (where applicable):

  • Appropriateness and quality of the proposed approach to ensure meaningful, safe and appropriate engagement plan with equity-denied communities throughout the proposed project

4. Potential impact and value-add to the applicant and the host partner organization (20%)

Evaluation Criteria Information Source Notes for Reviewers
a. Strength, clarity and relevance of the applicant's statement of motivation for applying, objectives and impact goals for the fellowship, professional development plan and their rationale for the three targeted enriched core competencies for development in relation to their career objectives.
  • All application materials, particularly the "Motivation for Applying and Value-Add" statement

Consider:

  • The relevance of the applicant’s motivation and commitment to the program objectives, how this opportunity will strengthen the applicant's capabilities and advance their ability to achieve said career goals
  • The strength and appropriateness of the applicant’s rationale for the top three enriched core competencies identified for development in relation to their fellowship and career objectives
  • Whether and how the experiential learning opportunity will add value to the applicant's doctoral/post-doctoral training (e.g., consider whether it provides exposure to a new organization, provides an opportunity to develop new skills and competencies, adds a new dimension to the applicant's doctoral/post-doctoral research, equips the applicant with the skills and relationships for a managerial/leadership position in the health system, etc.)
b. Potential impact and value-add to the organization of hosting the applicant (i.e., what will be achieved that could not without the applicant?). Explanation of how this award will add value to the organization (i.e., change, enhance, create new) and expand organizational research capacity is required.
  • Primary Supervisor's letter of support
  • Applicant's description of "Project/Program of work proposal"

Consider:

  • How hosting the applicant will add value to the organization, such as whether the applicant has expertise in a desired methodology, the applicant is tackling a complex and unsolved challenge faced by the organization, the applicant will build internal research capacity within the organization, the applicant will develop a needed program/database/tool or evaluate a key program/policy/tool, etc. What achievements will be possible with the applicant that could not be attainable otherwise? How will organizational research capacity be expanded?
Date modified: