Cochrane Corner Column

Reporting equity in systematic reviews: The Prisma equity extension

Vivien Runnels

Some populations (including women and girls, people living in poverty, and certain racial groups) have limited or unequal access to health interventions and experience differences in health outcomes. Health inequity refers to avoidable differences in health which are unfair and unjust. While systematic reviews are designed to assess and transfer information about the effectiveness of health interventions, they are also an important means of transferring knowledge about health inequities. Reporting equity in systematic reviews is critical for building an evidence base of interventions that address and justify action against inequities in health, including those that exist along sexed and gendered lines.

Although there is available guidance on conducting equity-focused systematic reviews, there has been no guidance on reporting them. Failing to report equity-focused reviews can inadvertently perpetuate health inequities by limiting our capacity to understand and address any differential effects of an intervention. Adding an equity 'lens' to widely endorsed reporting guidelines, such as the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) Statement, can have important implications for decision makers using systematic review evidence.

The Campbell and Cochrane Equity Methods Group developed reporting guidelines for equity-focused systematic reviews as an extension of the PRISMA Statement, called PRISMA-E 2012. The project had two main goals:

  1. to provide structured guidance on transparently reporting methods and results in equity-focused systematic reviews, and
  2. to legitimize and emphasize the importance of reporting health equity results.

PRISMA-E 2012 is designed at minimum to raise both systematic reviewers' and users' awareness of disadvantage and inequities, with sex and gender being one category across which disadvantage may exist. Equity extension items have been added to different sections of systematic reviews. For example, the standard PRISMA item for the Abstract calls for a structured summary, including background, objectives, data sources and other applicable components. The equity extension to the Abstract item expands on this by instructing authors to "present results of health equity analyses (e. g. Subgroup analyses or meta-regression)," focusing attention on specific steps in reporting that are needed to demonstrate equity or inequity.

PRISMA E-2012 was published in October in PLOS Medicine. The authors' next steps include disseminating the reporting guidelines to increase their uptake, evaluating the effects on reporting equity (including sex and gender) in prospective studies, contacting journal editors to request their endorsement of PRISMA-E 2012, and conducting an evaluation to determine whether journal endorsement of PRISMA-E 2012 impacts reporting of equity considerations in systematic reviews. We also hope that over time, requiring reporting of equity and sex and gender in systematic reviews may influence primary researchers to report more on study characteristics, provide sex disaggregated data, and reflect on the applicability of their findings. In addition, we hope that endorsement of these guidelines by journals will encourage and improve transparency and comprehensiveness of reporting of systematic reviews, therefore improving their relevance for clinical practice and policy making.

The Campbell and Cochrane Equity Methods Group and authors of the PRISMA Equity Extension sincerely thank CIHR and the Rockefeller Foundation for funding support.

Methods for Selecting and Summarizing Reviews

The IGH Cochrane Corner is a resource to promote considerations of sex and gender in systematic reviews. The Corner provides access to plain language summaries of systematic reviews that report on the strengths and weaknesses of the approaches to sex and gender used by review authors. This edition of the Column details the methods by which reviews are selected and summarized and how summaries are peer reviewed for the IGH Cochrane Corner.

Selection Process

In selecting reviews for inclusion in the Corner, we consider the following criteria:

  1. Fit with IGH's strategic directions.
  2. State of completion and date of publication.
    • We only select completed systematic reviews and prioritize those that are most recent.
  3. Impact on men and women.
    • Given that our interest is to analyze how reviews consider sex and gender in terms of reported details such as sample populations and outcomes, we limit review topics to issues that affect both men and women, rather than those that are sex specific.

Summaries: Sex and gender analysis

While authors include plain language summaries in their systematic reviews, these summaries often remain technical. The summaries created for the IGH Cochrane Corner are written so that specific topic knowledge is not required to understand the variety of reviews presented.

These summaries:

  • Introduce and define the topic of the review;
  • State the objectives of the review;
  • Identify the overall results from a sex and gender perspective;
  • Evaluate the approach to sex and gender used in the review; and
  • Assess the overall implications of the review findings in terms of sex and gender.

In reporting review results and evaluating the approach to sex and gender, our summaries address three questions related to sex and gender:

  1. How were sex and gender considered in the review?

    Considering sex and gender means looking at how a given intervention may affect men and women or males and females differently, or how sex and gender may have influenced the outcomes of the intervention. Sex and gender implications may be described in the background information provided in a review or through a subgroup analysis. This also involves assessing the applicability of evidence; if the studies used in the review included participants of only one sex or gender, its conclusions may not be universally applicable.

  2. What are the strengths and weaknesses of the approach to sex and gender?

    Here, we highlight the strengths of the authors' approach to sex and gender, as well as areas that could be improved upon. A common strength or weakness results from whether or not review authors distinguish between the terms 'sex' (biological) and 'gender' (socio-cultural). Other strengths result from the extent to which authors are able to present disaggregated findings, on the basis of not only sex or gender but other intersecting grounds as well. In some cases, the authors of systematic reviews intend to do a subgroup analysis by sex but are limited by a lack of data from primary studies.

  3. What do we know about sex and gender based on this review?

    This section is used to highlight implications for policy and practice as well as for future research in relation to health equity. Implications for policy and practice focus mainly on whether or not the systematic review supports the intervention as effective. Here, we also examine the different implications that findings may have according to sex or gender. However, if findings were inconclusive, further research may be recommended.

Peer Review Process

Before the summaries are published in the IGH Cochrane Corner, they are sent to the original review authors for consideration. This gives the authors the opportunity to respond to the summary and make recommendations.

Sensitizing Systematic Reviews for Sex/Gender, Equity and Bias: Some Challenges

Vivien Runnels, Marion Doull, Sari Tudiver and Madeline Boscoe

Systematic reviews are designed to evaluate the effectiveness of interventions in health and policy by gathering and analyzing evidence from multiple primary studies. Patients, practitioners and others consult systematic reviews for evidence-based recommendations that are applicable and context specific. However, systematic reviewers encounter challenges in acquiring and synthesizing data in relation to sex and gender, which in turn have implications for methods bias and equity in outcomes. We report here on some of these challenges synthesized from responses to a questionnaire developed by our Working Group on Sex and Gender in Systematic Reviews that was circulated to colleagues with interest and expertise in systematic reviews prior to the meeting "Combining Forces to Improve Systematic Reviews: Gender, Equity and Bias" (Ottawa, ON, May 2011).

Conceptual challenges

How a problem is framed, conceptualized and ultimately researched is determined by decisions made early in the systematic review process. Upstream decisions shape the production of knowledge including determining when to investigate sex and gender differences, when to factor in questions of equity and how to address potential issues of bias in trial design. However, these decisions can only be made on the basis of pre-existing knowledge. Conceptual challenges include lack of understanding the distinct, but interrelated concepts of sex and gender, lack of awareness about potential sex/gender-based differences and the importance of equity for health interventions, research and outcomes.

Methodological challenges

Assessing sex, gender, equity and bias in systematic reviews presents numerous methodological challenges. These include: poor quality protocols; lack of transparency of clinical trials and the resultant inability to determine whether outcomes and analyses were specified a priori; difficulties in completing robust subgroup analysis when there is under-representation of those from disadvantaged groups in the trial population and when published studies do not provide sex-disaggregated data. These limitations suggest a need for appropriate guidance and training for researchers, systematic reviewers and the broader research community (e.g. journal editors, publishers, funders).

From concepts to methods

Overall, colleagues emphasized that a lack of conceptual clarity is interconnected to, and iteratively feeds, methodological challenges. This laid the basis for discussions at the "Combining Forces" meeting where we identified a need for indicators for gender and other health determinants, and for guidelines for how to achieve quality sex/gender analysis within reviews and for theorizing how to interpret generalized data for diverse individuals and groups. Together, we pointed to the value of highlighting gaps in evidence even if appropriate sub-group analyses cannot be carried out and noted that a lack of tools, checklists, and/or outlines for conducting sex/gender analysis in systematic reviews creates a challenge for methodologists unfamiliar with the concepts of sex and gender in particular.

The conceptual and associated methodological challenges that the inclusion of sex, gender, equity and bias can present for systematic reviewers have limited the uptake of these issues and their associated analyses into reviews. Addressing the challenges by developing methods and tools to facilitate and build critical capacity has significant potential for enhancing the applicability and the quality of the evidence to achieve better health outcomes for all.

The authors thank the participants at the "Combining Forces" workshop whose thoughtful input allowed them to contribute this column. "Combining Forces" was funded through an IGH Meetings, Planning and Dissemination Grant.

Integrating Sex and Gender in Logic Models for Systematic Reviews

Erin Ueffing and Jordi Pardo Pardo

Logic models are visual representations of theories about how an intervention works in given contexts. Logic models have been suggested as tools to improve the understanding of whether an intervention works in a particular population: "Equity oriented systematic reviews should include a logic model to elucidate hypotheses for how the intervention (whether a policy or a programme) was expected to work, and how factors associated with disadvantage (social stratification) might interact with the hypothesized mechanisms of action".Footnote 1

One example of a logic model used in systematic reviews and practice guidelines is that developed by the US Preventive Services Task Force.Footnote 2 This logic model, or "analytic framework", illustrates the population, interventions, and outcomes to be considered in a systematic review. It can be adapted easily to map the influence of sex and gender on the various links between the population and the potential outcomes (Figure 1). For example, consider a community program for HIV/AIDS. Both sex and gender would be important considerations when identifying persons at risk: females are more susceptible to HIV than males, while women often have less sexual power or control than men. Sex and gender also play roles in risk groups such as commercial sex workers or men who have sex with men. For those who participate in screening programs, potential adverse effects vary by gender: women may be at higher risk than men for community exclusion, spousal violence or rejection, and family conflict both if they are screened and if they are diagnosed with HIV.Footnote 3 Treatment or interventions, too, may vary by gender. For example, many HIV/AIDS education and counselling programs are tailored by gender. In addition, many interventions intended to reduce HIV transmission are tailored by sex, such as female condoms. Finally, the intermediate outcomes that are chosen for evaluation may differ by sex. For example, the reduction in mother-to-child transmission of HIV would be an appropriate outcome only for those interventions aimed at females who are or may become pregnant.

As this example has shown, logic models can be used usefully and appropriately to map the potential influences of gender and sex from the population to the outcomes of an intervention. In systematic reviews, logic models such as this can serve a variety of purposes. For example, mapping gender and sex may justify the decision to limit a review to one sex or gender, or justify subgroup analyses to examine the differential effects of an intervention across sex and gender.Footnote 4

Logic models are powerful tools to illustrate how sex and gender play roles in complex interventions, and understand how sex and gender link with other factors to modify the effects of an intervention or the condition for which an intervention is intended.

Sex and Gender in the Cochrane Library: Looking Back, Moving Forward

Erin Ueffing and Jordi Pardo Pardo, Campbell and Cochrane Equity Methods Group

In the ten years since the inception of the Institute of Gender and Health, there have been increasing calls from policy makers, practitioners, and researchers for the integration of gender and sex in health research.Footnote 5 At the same time, there has been a movement toward evidence-based health care; that is, using available research evidence to guide the decision making affecting health care practices and patient care.Footnote 6 International groups such as the World Health Organization's Commission on Social Determinants of Health have identified systematic reviews as one form of evidence that meets these needs. The Cochrane Collaboration publishes systematic reviews of health interventions in The Cochrane Library.

Looking back over the past decade, three studies have investigated how reviews produced through the Cochrane Collaboration have considered the applicability of the research evidence assessed with regards to differences by sex or gender. Two of these surveys focused exclusively on sex or gender,Footnote 7 Footnote 8 while the third also assessed other sociodemographic factors such as race/ethnicity, socioeconomic status, and education.Footnote 9

The first survey, conducted by Johnson et al., examined 30 Cochrane reviews on heart, hypertension, and peripheral vascular diseases in 2001. Of these reviews, only three considered whether gender played a role in the effectiveness of interventions. Of the 196 studies that recruited both men and women and were included in these reviews, 65 (33%) examined outcomes by gender. Of the 65 trials that performed a gender-based analysis, 13 (20%) reported significant differences in cardiovascular-related outcomes by gender.Footnote 7

In their 2010 update of Johnson's study, Doull et al. examined 38 randomly selected reviews on heart, hypertension, and peripheral vascular diseases published since 2001. As with the Johnson study, Doull et al. found that differences between men and women were rarely considered; only two of the 38 reviews presented results disaggregated by sex or gender. The authors further noted that the terms "sex" and "gender" were used interchangeably, and that reviews often reported on the populations included in primary studies in terms of "% male", without describing the remaining participants.

Tugwell and colleagues examined how sex, gender, and a range of sociodemographic characteristics were accounted for in reviews on rheumatoid arthritis. The authors included all 14 reviews published between 2003 and 2008, which collectively reported on 147 primary studies. Five reviews (35.7% of the 14) and 131 primary studies (89.1% of the 147) reported the proportion of men and women included as participants at the start of the primary studies. Notwithstanding the share of primary studies reporting this information, Cochrane reviews on rheumatoid arthritis did not assess whether the effects of interventions for rheumatoid arthritis differed for men and women. The authors concluded that important differences might be missed and that systematic reviews can help to identify evidence gaps to guide future research.Footnote 9

These surveys show that across different time periods and subject areas, sex and gender are not considered or reported consistently in Cochrane reviews related to heart, hypertension, and peripheral vascular diseases and rheumatoid arthritis. Future work is needed to determine how Cochrane reviews on interventions in other areas of health fare in accounting for sex and/or gender. Moreover, these studies demonstrate that there are gaps in understanding sex and gender, from the definition of the concepts to the implementation of the review methods to analyze sex and gender. Cochrane reviews have not been sufficiently sensitive to results from gender- and sex-based analyses and may have underreported relevant findings.

Moving forward to tackle these knowledge gaps, the Campbell and Cochrane Equity Methods Group develops methods for adequately addressing sex- and gender-based analysis in systematic reviews. These methods help to determine when and how sex and gender should be considered in Cochrane reviews, and ways to address them in the analysis. The Equity Methods Group also works with Canadian and international colleagues to build the evidence base on the differential effects of interventions on the basis of sex and gender.

Visit IGH Cochrane Corner.


Footnote 1

Tugwell P, Petticrew M, Kristjansson E, Welch V, Ueffing E, Waters E, Bonnefoy J, Morgan A, Doohan E, Kelly MP. Assessing equity in systematic reviews: realising the recommendations of the Commission on Social Determinants of Health. BMJ. 2010 Sep 13;341:c4739. doi: 10.1136/bmj.c4739.

Return to footnote 1 referrer

Footnote 2

Harris RP, Helfand M, Woolf SH, Lohr KN, Mulrow CD, Teutsch SM, Atkins D; Methods Work Group, Third US Preventive Services Task Force. Current methods of the US Preventive Services Task Force: a review of the process. Am J Prev Med. 2001 Apr;20(3 Suppl):21-35.

Return to first footnote 2 referrer

Footnote 3

United Nations Population Fund (UNFPA). HIV Prevention Now Programme Briefs No.4 - Addressing Gender Perspectives in HIV Prevention. 2002. Available at the UNFPA website [ PDF (38 KB) - external link ].

Return to footnote 3 referrer

Footnote 4

Anderson L, Petticrew M, Rehfuess E, Armstrong R, Ueffing E, Baker P, Francis D, Tugwell P. Using Logic Models to Capture Complexity in Systematic Reviews. In press.

Return to footnote 4 referrer

Footnote 5

Sen, G. and Östlin, P. (2008) 'Gender inequity in health: why it exists and how we can change it', Global Public Health, 3:1, 1 - 12.

Return to footnote 5 referrer

Footnote 6

Dawes M, Summerskill W, Glasziou P et al. Sicily statement on evidence-based practice. BMC Med Educ 2005; 5: 1– 7.

Return to footnote 6 referrer

Footnote 7

Johnson SM, Karvonen CA, Phelps CL, Nader S, Sanborn BM. Assessment of analysis by gender in the Cochrane reviews as related to treatment of cardiovascular disease. J Womens Health (Larchmt) 2003;12(5):449-57. Available: PM:12869292.

Return to footnote 7 referrer

Footnote 8

Doull M, Runnels VE, Tudiver S, Boscoe M. Appraising the evidence: applying sex- and gender-based analysis (SGBA) to Cochrane systematic reviews on cardiovascular diseases. J Womens Health (Larchmt ) 2010;19(5):997-1003. Available: PM:20384450.

Return to footnote 8 referrer

Footnote 9

Tugwell P, Maxwell L, Welch V, Kristjansson E, Petticrew M, Wells G, et al. Is health equity considered in systematic reviews of the Cochrane Musculoskeletal Group? Arthritis Rheum 2008;59(11):1603-10. Available: PM:18975366.

Return to footnote 9 referrer

Date modified: