Expert Review Team Report for Institute of Infection and Immunity

Table of contents

• Summary
• Section 1 – Institute mandate
• Section 2 – Status of this area of research in Canada
• Section 3 – Transformative Impacts of the Institute
• Section 4 – Outcomes
• Section 5 – Achieving the Institute mandate
• Section 6 – ERT observations and recommendations
• Appendix 1 - Expert Review Team
• Appendix 2 - Key Informants

Summary

Report Headlines

Overall impression of the Canadian research landscape in this area:

• Strong activity with many highpoints of international excellence

III has been transformative in a number of areas:

• training and capacity building
• researcher career development in all pillars but notable new activity in pillars 3 and 4
• recruitment of partners to cross-cutting institute initiatives to address research priorities
• engagement with wide range of stakeholders to determine priorities
• rapid response to emerging infections (practice and policy)
• development and roll-out of specific community-based research activities

III pillars 3 and 4 have generated positive and productive outcomes:

• Noteworthy outcomes are more difficult to judge under pillars 1 and 2, especially outside the realm of the HIV/ AIDS Research Initiative

III has achieved its mandate in part:

• Successful and energetic leadership has been achieved at the national level
• Priorities have been set and programs funded to carry out research that will reduce the global burden of infection and immune-based diseases

Overall impression:

• There are many successes of which the Institute of Infection and Immunity (III) can be proud – the challenge for the next funding period is to increase synergy across the four pillars and effectively prioritise research so that the III continues to be more than the sum of its parts.

To note: The Expert Review Teams (ERT) for the 2011 CIHR International Review were asked to provide an objective assessment of the effectiveness by which the institutes are fulfilling their mandate. The ERT for III found this to be a challenging brief, given the relative brevity of and lack of depth in the Internal Assessment Report, together with the limited supplementary information provided in advance (lists of grants and key publications with appropriate attributions) that were not transparently linked to III output. We advocate generation of standard metrics that would allow robust assessment of III's contribution to advancing knowledge, building capacity and achieving impact within their target areas of activity. Our face-to-face discussions on the day of the review were open, interactive and informative, allowing us to gain considerable insight into the operations and achievements of the III under its Scientific Director.

Section 1 – Institute mandate

Created in 2000 as one of the 13 institutes of the Canadian Institutes of Health Research (CIHR), the Institute of Infection and Immunity has a mandate to support research to enhance immune-mediated health and to reduce the burden of infectious disease, immune-mediated disease and allergy through prevention strategies, screening, diagnosis, treatment, support systems and palliation. The III mandate transcends disciplines and encompasses all four health research themes: biomedical; clinical; health systems and services; and social, cultural and environmental factors that affect the health of populations. III's mission is to establish national leadership, priorities and programs to reduce the global burden of infection and immune-based diseases.

CIHR Institute of Infection and Immunity – Internal Assessment for 2011 International Review, pg 1

Section 2 - Status of this area of research in Canada

Infection and immunity research in Canada covers a range of activities that span from the highest internationally-ranked science to areas of national priority, incorporating the four prioritised themes (or pillars): biomedical, clinical, health systems and health services, health of populations, societal and cultural dimensions. The III operates within this arena, drawing extensive additional strategic funding (HIV/AIDS, Pandemic Preparedness etc) on top of its core budget, with III-affiliated researchers also securing extensive funds from the Open Operating Grant Program.

The HIV/AIDS Research Initiative has a large budget ($22.5M/year but leveraging substantial additional support externally e.g. Bill and Melinda Gates Foundation funding) and its own governance. It funds in all 4 pillars with substantial research at the leading edge internationally. Highlights include the identification of a T cell–specific HIV reservoir critical in viral persistence (pillar 1), the launch of a Canadian HIV Vaccine initiative and a Canadian HIV Trials Network (pillar 2) and the development of dual streams of community-based research (CBR) in the general and aboriginal populations, perhaps most notably studying the relationship between housing status, housing stability, health outcomes for people living with AIDS (pillars 3 and 4).

High profile public health threats such as SARS and H1N1 have increased the visibility and public awareness of III. For SARS, the leadership and rapid response of the III teams was universally admired, with subsequent establishment of the Canadian Rapid Research Response Team, a framework for rapid vaccine clinical trials, and the development of new guidelines and practices for international roll-out.

Other areas of III strength include developing activity in antibiotic resistance, including partnership working internally and externally (with the UK Medical Research Council joint funding initiative), as evidenced by increased publications/citations. Newer initiatives include world-leading microbiome studies and vaccine research while the immunity balance will be maintained and strengthened by new research planned in the areas of transplantation and inflammation.

Overall impression of the Canadian research landscape in this area

• Strong activity with many highpoints of international excellence.
• Good evidence of progression across all four pillars although some funding tensions apparent during consultations.
• CBR efforts in the HIV/ AIDS initiative are highlighted, with the role of strong leadership and availability of adequate resources noted.
• The microbiologically Safe Food and Water Initiative is also highlighted for its success, although it has been challenging to monitor outcomes to date.
• Maintaining a balance of infection and immunity research is desirable within the III mandate.
• Going forward, better metrics are essential to validate success in all areas and attribute this to the III and/or other funding agencies.

Section 3 - Transformative Impacts of the Institute

• Strong focus on training and capacity building via the use of Strategic Training Initiative in Health Research programs to develop the next generation of young researchers (noted in pillars 3 and 4). Training also very important in pillars 1 and 2 but lack of posts afterwards reported as a disincentive for initial recruitment and career development (suggested measures from participants: restitution of mid-/senior-career awards; resolution of recruitment issues surrounding Canadian Research Chairs).
• CBR activities under pillars 3 and 4 in HIV/AIDS and Safe Food and Water Initiative; will impact on policy in the mid-term.
• Speed and responsiveness of III to SARS and H1N1; have set international standards of excellence in practice and policy (for Canadians and roll-out internationally) for dealing with emerging infectious agents.
• III has helped to shape the establishment of a network and communities in Canada which foster interactions among researchers working under all four pillars.
• Limitations:
  • III research priorities regrouped between the 2002-2007 and 2007-2012 strategic plans to achieve greater focus: it is not clear that this has been achieved from the evidence provided.
  • III and stakeholders should explore the opportunity to develop a research, pandemic preparedness strategy based on deployment of anti-virals (effectiveness, economics, storage etc).

Overall impression – to what extent has this Institute been transformative?

III has been transformative in a number of areas:

• training and capacity building
• researcher career development in all pillars but notable new activity in pillars 3 and 4
• recruitment of partners to cross-cutting institute initiatives in order to address research priorities
• engagement with wide range of stakeholders to determine III priorities
• rapid response to emerging infections (practice and policy)
• development and roll-out of specific CBR activities

Section 4 - Outcomes

• III has catalyzed and built on priority areas as evidenced by the relative publications of select fields (limited bibliometric data provided for Antibiotic Resistance and Infection Control).
• III's CBR (pillars 3 and 4) research programs have increased awareness of and approaches to Aboriginal health and social-economic issues.
• Limited evidence on the preparedness to pandemics was presented but the outcome of SARS has led to global policy change.
• III uses "end of grants" and "mid-term/progress" reports to summarise research highlights and consequently, to develop or align new strategic initiatives. The extent and scope of these reports were unclear.
• Practical knowledge translation (KT) activities were not evident in several areas while successes were neglected (in the report but not during the review sessions) in others e.g. successful launch of spin-out companies in Quebec.

Overall impression – to what extent has this Institute been successful in achieving outcomes?

• Pillars 3 and 4 have generated positive and productive outcomes.
• It is more difficult to judge noteworthy outcomes under pillars 1 and 2, especially outside the realm of the HIV/ AIDS Research Initiative. Better evidence is needed for robust assessment, although the limited list of high-ranking papers provided in the III report is indicative of impressive basic science at the cutting-edge internationally.
• KT activities are strong in diagnostics (although not covered in the III report).

Section 5 - Achieving the Institute mandate

Overall, the III is respectful of and responsive to its mandate. It supports research across the breadth of infection and immunity and determines strategic priorities through broad discussion and consolidation across its full range of stakeholders. The Scientific Director is the ultimate arbitrator of funding decisions, backed by the Institute Advisory Board and external review. It is unclear as to the relative contributions of each of these or whether decisions can be appealed. Are views of other stakeholders equally valued?

The III works across all four health research pillars and is highly effective in this activity in some areas; in others, there is less obvious synergy between pillars 1 and 2 and pillars 3 and 4. This is a challenge to be addressed.

Other limitations:

• III should encourage reasonable "entrepreneurial" activities (e.g. intellectual property (IP) protection) in order to promote the development of novel products, services and/or training materials (sponsor commercial value, job creation and knowledge-based economy).
• Disproportionate funding envelope for HIV/AIDS research in Canada; can be difficult to justify interest in one infectious disease in light of underfunded/neglected areas.
• More support is needed to fund vaccine development and safety independently from industry. Need for surveillance and monitoring of vaccines.
• KT remains underwhelming.

Overall impression – to what extent has this Institute achieved its mandate?

• Successful and energetic leadership has been achieved at the national level.
• Priorities have been set and programs funded to carry out research that will reduce the global burden of infection and immune-based diseases.
• Going forward, and given the stated aim of funding fewer, larger programs, the robustness and transparency of strategic priority setting should be reinforced, within an international context.

Section 6 - ERT Observations & Recommendations

• The III internal assessment report lacked specificity and sufficient quantitative data:
  • Should address whether goals were met and lessons learnt – metrics of success.
  • Should contain a strengths, weaknesses, opportunities & threats (SWOT) analysis as an essential element in forward planning.
  • Should stress interdisciplinary connections (physics, chemistry, math etc.) where these exist (absent from report).
  • Should be accompanied by robust data that allow tracking of III investment and achievement across the whole remit.
• Communication of III activities to the scientific community, partners and the general public is a recommended area for improvement (III website should be highly interactive).
• Linkages with industry can be critical for translational activity. Evidence for varied but substantial industry/III connections (mostly from discussions) but quantitative data lacking (e.g. how many companies, level of activity, co-funding, patents, IP).
• Peer review exhaustion is not unique to Canada nor to III's strategic initiatives but extends to the global scientific community.
• Key issue is sustainability- how to sustain funding for new and existing strategic initiatives.
• Comments to review team indicate "review fatigue" – it was suggested by participants that the costs involved could be better spent on developing international partnerships. The ERT disagree but suggest that the collection and use of better metrics would facilitate more robust and cost-effective review processes.

Overall impression of the performance of this Institute

• Recommendations outlined in the 2006 International Review have been only partially addressed: KT has improved (although there is more to do); communications have improved and will continue to do so; the identification of effective performance targets at an Institutional level is not apparent.
• The Scientific Director has successfully maintained cohesion in the community at large to date (he has been in post for ~1 year) and readily seeks stakeholders input. The Scientific Director is also clearly dedicated to his role in building relationships with the research community.
• Given the modest size of III's $ 8.5M core budget, strategic initiatives have an overall good return on investment as could be judged from written and spoken testimony.
• Testimonials were heavily focused on HIV/ AIDS; other III initiatives received less attention in discussion.
• Overall, there are many successes of which III can be proud – the challenge for the next funding period is to increase synergy across the four pillars and effectively prioritise research so that the III continues to be more than the sum of its parts.

Recommendations

• III should instil an "entrepreneurial" spirit in its staff and endeavours.
• Further investment in the field of "Diagnostics" should be prioritised.
• III should seek to strengthen linkages between CFI "infrastructure" and CIHR "program" grants. Funding and support of the infrastructure to underpin the highest-quality research activity are global problems; integrative solutions will accelerate Canadian research activity and may roll out to the wider research community.
• Recommendation to include/ consult with the international community in most if not all decision-making processes (depending on the area/ pillar; Aboriginal issues may be best addressed by Canadian experts).
• Tracking the progress and productivity of III funding recipients and monitoring success stories should be mandatory and fully implemented for future reporting purposes.

Appendix 1 - Expert Review Team

Chair - Professor Deborah Smith
Professor of Molecular Parasitology
Centre for Immunology and Infection, University of York
Chair, MRC Infections and Immunity Research Board, UK

Expert Reviewer – Professor Hidde Ploegh
Professor, MIT Department of Biology
Whitehead Institute for Biomedical Research
Cambridge, MA USA

International Review Panel – Professor Rudi Balling
Director - Luxembourg Centre for Systems Biomedicine
University of Luxembourg

Appendix 2 - Key Informants

Session 1 – Review of Institute

1. Dr. Marc Ouellette, III Scientific Director
   
2. Dr. Chris Power, Chair – Institute Advisory Board
   Professor, Departments of Medicine and Medical Microbiology & Immunology
   University of Alberta
   
3. Dr. Katherine Siminovitch
   Head, Division of Genomic Medicine
   Toronto General Research Institute
   Senior Investigator, Samuel Lunenfeld Research Institute
   Mount Sinai Hospital
   
4. Dr. Martin Schechter
   Professor, School of Population and Public Health
   University of British Colombia

Session 2 – Consultation with researchers

1. Dr. Keith Fowke
   Associate Professor, Department of Medical Microbiology
   University of Manitoba
   
2. Dr. Sean B. Rourke
   Assistant Professor, Department of Psychiatry
   University of Toronto
   Scientific and Executive Director
   Ontario HIV Treatment Network
   
3. Dr. Michel Bergeron
   Director, Division of Microbiology and Le Centre de Recherche en Infectiologie
   Université Laval

Session 3 – Roundtable with stakeholders

1. Dr. Mike Mulvey
   Chief, Antimicrobial Resistance and Nosocomial Infections
   National Microbiology Laboratory
   Public Health Agency of Canada
   
2. Dr. Chris Archibald
   Director, Surveillance and Risk Assessment Division
   Public Health Agency of Canada
   
3. Dr. Arlene King
   Chief Medical Officer of Health
   Ontario Ministry of Health and Long Term Care
   
4. Dr. Neil Cashman
   Scientific Director
   PrioNet Canada

Date modified:

2011-08-02