Beyond the pyramid: Reconsidering evidence synthesis as distinct from original research

Systematic reviews and meta-analyses: Syntheses, not discovery

Systematic reviews and meta-analyses are secondary methods that organise and combine existing study results but do not generate new data. They depend on the quality of the original studies, and treating them as equivalent to primary research risks conflating aggregation with discovery. The evidence pyramid also overlooks emerging methods like qualitative syntheses and realist reviews, which are essential for understanding complex interventions.^3,4 These approaches provide critical insights but have often been excluded from traditional hierarchies.

While meta-analysis is the most recognised method, others like realist reviews and meta-ethnography also play crucial roles, especially for complex health interventions. Realist reviews explore how, why, and for whom interventions succeed or fail,⁵ while qualitative syntheses inform public health guidelines when RCTs aren’t feasible.^6,7 Overemphasizing meta-analysis risks ignoring patient perspectives and contextual factors, which are essential for policy-making. To ensure comprehensive, context-sensitive decisions, policy-makers should embrace a variety of synthesis methods.^8,9 Recent experience underscores this point: during the COVID-19 pandemic, early meta-analyses of hydroxychloroquine based on small, low-quality trials suggested benefits and informed policy, yet were later overturned by more rigorous studies.¹⁰ This example illustrates how over-reliance on quantitative syntheses can mislead when the underlying evidence is weak, highlighting the need to also consider qualitative insights and other review types.

Disadvantages of the current evidence pyramid

The emphasis on evidence synthesis in recent years is reflected in publication trends: by 2019, nearly 30,000 systematic reviews were being indexed annually (about 80 per day), a more than 20-fold increase since 2000. The evidence pyramid model has encouraged desk-based evidence synthesis where systematic reviews and meta-analyses are conducted without original data collection.¹¹ These secondary studies, often pursued for academic credit or citations, promote evidence recycling with marginal updates.¹² Multiple overlapping reviews on the same topics have also become common, contributing to confusion and research waste in the literature.¹³ International collaborations now focus on synthesis consortia, weakening the infrastructure for primary data collection and original research.¹⁴ Evidence Based Medical education and journal and author metrics prioritise meta-analyses, fostering the false belief that synthesis equals strength – an attitude which ultimately undermines original research and innovation.¹⁵

Proposing a revised dual-framework model

We propose a dual-framework model (Figure) to supplement the single-hierarchy pyramid. One side recognises the hierarchy of original, evidence-generating research, while the other side values evidence integration (synthesis) as a distinct but equally important process. In the evidence generation hierarchy, well-conducted cluster RCTs and large multicentre trials are at the top, followed by other quantitative research designs and, finally, rigorous qualitative studies that provide contextual insights. This model emphasizes methodological alignment and pluralism,^16,17 inclusive of qualitative and mixed-method research as essential for understanding contexts where RCTs may not suffice, e.g., regarding implementation, mechanisms, or patient experience. Importantly, we give methods like qualitative evidence syntheses and realist review a place in the framework (within the synthesis oval) rather than omitting or subordinating them. This acknowledges ongoing debates about how to ‘rank’ such evidence and affirms that they contribute complementary knowledge that a linear hierarchy cannot capture. Figure illustrates this proposed Generation–Integration framework. The left pyramid displays a revised hierarchy of original research designs (i.e., studies generating evidence). The levels of the pyramid are defined by study design and methodological rigor – for example, well-designed RCTs and systematic observational studies toward the top, and exploratory or hypothesis-generating studies (including qualitative research) at the base. The right side is depicted as an oval encompassing evidence synthesis methodology, which integrates evidence into knowledge or policy. These synthesis methods (e.g., meta-analyses, realist reviews, qualitative evidence syntheses) are not placed within the pyramid but connected conceptually to reflect their integrative role outside a strict hierarchy. In this depiction, each synthesis method is considered a tool to be used as appropriate, rather than a ‘level’ of evidence quality.

Close

Figure.

Proposed Generation–Integration framework for evidence–based research.

Export to PPT

Advantages of the proposed model

By separating original data-generating research from synthesis methods, our model corrects the epistemological conflation that has misdirected research priorities. By epistemological conflation, we mean the blurring of evidence generation (discovery of new data) with evidence synthesis (aggregation of existing data) as if they were the same level of knowledge. This conflation is a problem because it can divert attention and resources toward redundant reviews at the expense of needed original research. Our dual-framework reorients value toward innovation, exploration, and rigorous design, rather than aggregation for its own sake. In doing so, it explicitly distinguishes creation of new evidence from integration of evidence, echoing prior calls to increase the yield of valuable research and reduce wasteful duplication. Funders and institutions can use this dual framework to balance resource allocation, supporting both high-quality original studies and meaningful evidence syntheses. In practical terms, this means funding agencies and policymakers might ensure that novel primary research is not starved in favour of easier secondary projects, instead investing in new trials or cohort studies where evidence gaps exist, while still supporting syntheses that address genuine needs. The model’s non-hierarchical oval depiction of synthesis methods promotes methodological equity, recognizing the value of meta-analyses, realist reviews, and qualitative syntheses based on their appropriateness for the question at hand rather than their position on a fixed ladder.

Overall, systematic reviews and meta-analyses should be viewed as valuable interpretive frameworks, but not the pinnacle of research. A reformed model that distinguishes original research from synthesis processes offers greater clarity, inclusivity, and alignment with modern health research goals is proposed. By embracing this balanced dual-framework approach, the medical fraternity can improve decision making at all levels from education and research funding to clinical guideline development ensuring that both the creation of new evidence and the synthesis of existing evidence are optimally utilised for patient care.