Translate this page into:
Comparative efficacy of leading COVID-19 vaccines: A network meta-analysis
For correspondence: Dr Shashank Tripathi, Department of Biostatistics and Medical Informatics, University College of Medical Sciences and GTB Hospital, Delhi 110 095, India e-mail: shashanktripathi031@gmail.com
-
Received: ,
Accepted: ,
Abstract
In the fight against the COVID-19 virus, various vaccines using different technologies such as mRNA, viral vectors, protein subunits, and inactivated whole viruses have become primary defence strategies. This study aims to compare their effectiveness in controlling the spread of the pandemic. Using the comprehensive resources from three major databases-PubMed, EMBASE, and the Cochrane Library-we conducted an extensive literature review up to April 30, 2023. By employing a frequentist network meta-analysis, we analysed both direct and indirect estimates of vaccine efficacy, providing a clear comparison of the leading candidates in the global fight against COVID-19. Fifteen vaccines from 26 articles were used in our network meta-analysis. The statistically significant direct estimates were obtained by Spikevax [VE: 93.29 (91.31, 95.27); P<0.05], Pfizer BioNTech [VE: 92.07 (90.03, 94.12); P<0.05], Sputnik [VE: 91.60 (85.60, 97.60); P<0.05], Novavax [VE: 88.99 (83.55, 94.42); P<0.05], Sinovac [VE: 83.50 (65.40, 101.60); P<0.05], Covifenz [VE: 77.27 (68.48, 86.06); P<0.05], Zifivax [VE: 75.94 (70.86, 81.02); P<0.05], Covishield [VE: 72.34 (67.12, 77.56); P<0.05], S-Trimer [VE: 71.61 (56.23, 86.98); P<0.05], Covaxin [VE: 70.81 (65.33, 76.29); P<0.05], Soberna [VE: 69.70 (56.50, 82.90); P<0.05], Zydus Cadila [VE: 66.60 (47.60, 85.60); P<0.05], CVnCoV [VE: 63.70 (52.20, 75.20); P<0.05], Convidecia [VE: 57.50 (39.70, 75.30); P <0.05], and Jcovden [VE : 52.42 (47.28, 57.57); P<0.05]. Spikevax emerged triumphant with an unparalleled P score of 0.95, solidifying its status as a top ranking prevention tool against the COVID-19 in our investigation. Our analysis reveals a ranking of vaccine efficacy, with Spikevax emerging as the most effective, followed closely by Comirnaty, Sputnik, and others, collectively providing strong protection against the ongoing threat of COVID-19.
Keywords
COVID-19
direct and indirect estimates
efficacy
network meta-analysis
systematic review
vaccines
The Coronavirus Disease 2019 (COVID-19) was declared as a pandemic in March 2020. The world saw the rapid spread of its causative agent, the Severe Acute Respiratory Syndrome Coronavirus (SARS-CoV-2), followed by the loss of precious lives1 and disrupted societies. These were among the major concerns of the virus outbreak1. The first case of SARS-CoV2 was reported from Wuhan, China, in December 20192. The pandemic led to 52 million mortalities from 252 million infected global citizens. The big dip of national economies into recession3 was one of the major disappointments caused by lockdowns across the globe. Leading nations across the globe began racing to develop effective vaccines. In less than four years, approximately 700 vaccines were in different phases of development, and various regulatory authorities approved 37 vaccines. The Emergency Use Authorization (EUA) for seven vaccines was given in different nations4.
Vaccines save numerous lives and avert millions of illnesses annually5. The fundamental of vaccines is to introduce a harmless virus (dead or weakened) or its parts into the human body, which stimulates and trains the immune system6. The mRNA, virus vector, protein subunit, and whole virus-inactivated technologies were used to develop different vaccines7. The mRNA vaccines instruct cells to make the S protein (found on the surface of the SARS-CoV-2 virus). The S protein pieces are made by muscle cells, in response to which antibodies are created and displayed on the cell surface7. The vector vaccines were developed by affixing modified versions of different viruses (viral vectors) with materials from the SARS-CoV-2 virus. The S protein is made by cells after receiving instructions from the viral vector8,9. The immune system starts creating antibodies and defensive white blood cells (WBCs) once the S protein is displayed on the surface of the cell8,9. The protein subunit vaccines contain harmless S-proteins recognised by the immune system to create antibodies and defensive WBCs10,11. Whole virus vaccines (killed or attenuated) cause the human body to trigger protective immunity in response to deactivated forms of pathogens12.
Unlike traditional pairwise meta-analysis, which compares two treatments, Network Meta Analysis13 enables simultaneous analysis of several interventions, providing a comprehensive assessment of the outcome of interest. This approach is particularly valuable in fields with numerous treatment options, as it ranks treatments according to the outcome of interest. Network meta-analysis improves decision-making by synthesising a comprehensive evidence base, enabling the evaluation of multiple interventions simultaneously. It identifies the most effective options, even in the absence of direct head-to-head comparisons, providing a robust framework for comparing treatments and guiding evidence-based choices in complex healthcare or research scenarios14.
The vaccines against COVID-19 are developed by mRNAs, vectors, protein subunits, and whole-inactivated viruses; we designed an investigation to compare the efficacy of dissimilar vaccines developed across the globe in stopping the spread of COVID-19. The vaccines were also ranked based on their efficacy in the current investigation. Further, there are fewer superiority or non-inferiority trials comparing the inter-efficacy of various vaccines. Therefore, the current investigation provides significant efficacy with direction (positive or negative) of multiple comparisons of COVID-19 vaccines using network meta-analysis.
Materials & Methods
Design
We adhered to the guidelines of Preferred Reporting Item for Systematic Review and Meta Analysis-Network Meta-Analysis15. The protocol of the current network meta-analysis was registered in PROSPERO (CRD42023465300).
Search strategy
The two investigators (ST and SR) independently searched the three databases (PubMed, EMBASE, and Cochrane Library) till April 30, 2023, to identify the randomised controlled trials (RCTs) to provide the efficacy of the COVID-19 vaccine (all types). Any disagreement between the authors was resolved by regular meetings and debates utilising scientific pieces of evidence.
The key words used in above mentioned databases were “vaccines” OR/AND “AstraZeneca” OR/AND “Covishield” OR/AND “Convidecia” OR/AND “Covaxin” OR/AND “Covifenz” OR/AND “CVnCoV” OR/AND “Janssen” OR/AND “Jcovden” OR/AND “Novavax” OR/AND “Comirnaty” OR/AND “Sinovac” OR/AND “Soberna” OR/AND “Spikevax” OR/AND “Sputnik” OR/AND “S-Trimer” OR/AND “Zifivax” OR/AND “ZyCoV-D” OR/AND “ZyCoV-D” OR/AND “mRNA-1273” OR/AND “SCB-2019” OR/AND “ChAdOx1 nCoV-19” OR/AND “ZF2001” OR/AND “NVX-CoV2373” OR/AND “BBV152” OR/AND “AZD1222” OR/AND “BNT162b2” OR/AND” CoVLP+AS03” OR/AND “Ad5-nCoV” OR/AND “NVX-CoV2373” OR/AND “rAd5” OR/AND “SOBERANA-02” OR/AND “BNT162b2” OR/AND “Ad26.COV2.S” or combinations of these terms.
The inclusion criteria for articles were following PICOS16 guidelines: P (Population), participants enrolled in COVID-19 RCTs, I (Intervention), any COVID-19 vaccine administered as a preventive measure against COVID-19, C (Comparator), the placebo or the standard non-COVID vaccine given to the participants in RCTs, O (Outcome), the vaccine efficacy of the COVID-19 vaccines compared to the placebo or standard vaccine arm. In this review under S (Study design), we considered only the phase 2b/3 and 3 RCTs to estimate the efficacy of the different COVID-19 vaccines. Similarly, the exclusion criteria for articles were: (i) not conducted in humans and used a non-placebo group; (ii) the efficacy of the vaccines was not reported; (iii) the data to estimate efficacy were not reported; and (iv) articles not published in English or lacking translation.
Outcome
The vaccine efficacy estimated by (1-RR) *10017, reported from primary RCTs, was the main outcome used in the current network meta-analysis. The efficacy of vaccines reported against different COVID-19 variants, separately in single primary RCTs, was also considered.
Qualities of studies
The Oxford Scoring system (JADAD score)18 was used to assess the methodological quality of RCTs. The scoring was done in three domains: randomisation (2 questions), blinding (2 questions), and accountability (1 question). The score ranged from zero to five. Here, a score more than equal to three suggested that the study was of high quality, i.e., low risk of bias.
Data extraction
MS Excel was used to identify and screen articles. The two investigators (SR and ST) independently reviewed the articles for inclusion and extracted data for network meta-analysis. The data were extracted with the following headings: (i) last name of the author of the study (year of publication), (ii) country of the participants enrolled in the study, (iii) chemical names of the vaccines (brand name of the vaccine was also extracted), (iv) type of vaccine, (v) number of participants enrolled in treatment, (vi) number of participants enrolled in placebo or controlled arm, (vii) the variants on which vaccine efficacy was reported, and (viii) the vaccine efficacy.
Statistical analysis
We used Q-statistics and I2-statistics to estimate heterogeneity in the effect size across and within all the studies. The effect sizes in meta-analysis vary from study to study; therefore identifying and quantifying this heterogeneity is an important point to be considered. The Q-statistics examine the presence or absence of heterogeneity, whereas the I2 statistic describes the magnitude of heterogeneity. Based on these two measures of heterogeneity (Q and I2), the appropriate model (fixed effect model and random effect model) was selected to generate a pooled effect size. If the degree of heterogeneity in effect size was significantly high (i.e., I2>30%), a random effect model was used; otherwise, a fixed effect model was used19.
The direct and indirect estimates, by frequentist network meta-analysis, was obtained by selecting all the vaccines in the network as reference treatments simultaneously. The P score20 was estimated to rank the vaccines according to their efficacy.
We constructed forest plots for each design in the network separately to display the comparison of vaccines with the selected reference treatment. The publication bias was checked by both, the graphical method (funnel plot) and the mathematical method (Egger’s test).
The ‘netmeta’ package was used to estimate all effect sizes and construction of all the plots in the current investigation from the R (4.3.1.; R Foundation for Statistical Computing, Vienna, Austria).
Results
Characteristics of studies
A total of 1,200 articles were initially identified across three databases: PubMed (550), Embase (300), and the Cochrane Library (350), using the specified keywords. Following the removal of 350 duplicate records, an additional 170 were excluded by an automated tool through title and abstract screening. This process yielded 680 articles for preliminary review. Of these, 380 were excluded as they did not align with the study’s objectives by reading title and abstracts. Subsequently, 300 articles underwent full-text screening, resulting in the exclusion of 267 that failed to meet the study’s inclusion criteria, following a different study design. Ultimately, 33 articles were deemed eligible for investigation under rigorous inclusion standards, and 26 of these were selected for inclusion in the final network meta-analysis. The whole search strategy is displayed in the PRISMA Diagram (Fig. 1).
- The PRISMA flow chart for inclusion of studies for network meta-analysis. The total 1200 articles were identified from databases. Only 680 articles were screened from which 300 articles were sought for retrieval and rest 380 articles were excluded from investigation. At last, 26 RCTs were included for network meta-analysis.
The efficacy of JCovden was reported by Sadoff et al21 (2022) on 11 different COVID-19 variants. Covishield efficacy by Clemens et al22 (2021) on five different variants, Emary et al23 (2021) on two different variants, Falsey et al24 (2021) and Voysey et al25 (2021) reported single efficacy on all types of variants. Covaxin was reported by Ella et al26 (2021) on four variants. CVnCov was reported by Kremsner et al27 (2021) on five types of variants. Comirnaty single efficacy was reported by Frenck et al28 (2021), Polack et al29 (2020), Walter et al30 (2021) on all types of variants. Thomas et al31 (2021) reported Comirnaty efficacy on two types of variants. Novavax single efficacy was reported by Dunkle et al32 (2021) on all types of variants, whereas Heath et al33 (2021) reported efficacy of same vaccine on three types of variants. Spikevax single efficacy was reported by Ali et al34 (2021), Baden et al35 (2022), Creech et al36 (2022), and Sahly et al37 (2021) on all types of variants. Zifivax was reported by Dai et al38 (2022) on four types of variants.
Convidecia, Sinovac, Soberna, Sputnik, and ZyCoV-D single efficacy was reported by Halperin et al39 (2021), Hager et al40 (2022), Tanriover et al41 (2022), Toledo-Romani et al42 (2022), Logunov et al43 (2021), and Khobragade et al3 (2022) on all types of variants. Covifenz efficacy was reported by Hager et al40 (2022) on four types of variants. At last, S-trimer was reported by Bravo et al44 (2022) on four types of variants. Therefore, a total of 59 studies were utilised in network meta-analysis. The detailed characteristics of selected articles, vaccines reported, and the respective efficacies are presented in table3,21-45.
| S. No. | Study (yr) | Country | Chemical name of vaccine (vaccine name) | Type of vaccine | Number of participants in treatment arm | Number of participants in standard arm | Variant | VE (95% CI) |
|---|---|---|---|---|---|---|---|---|
| 1 | Khobragad et al3 (2022) | India | ZyCoV-D (ZyCoV-D) | Protein subunit | 13 851 | 13 852 | B.1.617.2 (delta) | 66.6 (47.6-80.7) |
| 2 | Ali et al34 (2021) | USA | mRNA-1273 (Spikevax) | mRNA | 2489 | 1243 | All types | 92.7 (67.8-99.2) |
| 3 | Baden et al35 (2022) | USA | mRNA-1273 (Spikevax) | mRNA | 15,181 | 15,170 | All types | 94.1 (89.3-96.8) |
| 4 | Bravo et al44 (2022) | Belgium, Brazil, Colombia, Philippines, & South Africa | SCB-2019 (S-Trimer) | Protein Subunit | 15 064 | 15 064 |
Delta variants (B.1.617.2) Gamma (P.1) Mu (B.1.621) Other variants |
78.7 (57·3-90·4) 91.8 (44·9-99·8) 58.6 (13·3-81·5) 55 (24·9-73·8) |
| 5 | Clemens et al22 (2021) | Brazil & UK | ChAdOx1 nCoV-19 (Covishield) | Vector | 4772 | 4661 |
B.1.1.28 B.1.1.33 P.2 (Zeta) P.1 (Gamma) Other variants |
72.6 (46.4-86) 88.2 (5.4-98.5) 68.7 (54.9-78.3) 63.6 (−2.1, 87) 56.6 (28.2-73.8) |
| 6 | Creech et al36 (2022) | USA & Canada | mRNA-1273 (Spikevax) | mRNA | 3012 | 1004 | B.1.617.2 (delta) | 88 (70-95.8) |
| 7 | Dai et al38 (2022) | South east Asia | ZF2001 (Zifivax) | Protein Subunit | 12,625 | 12,568 |
B.1.617.2, AY.4, AY.6, or AY.12 (delta) B.1.1.7 (alpha) B.1.617.1(kappa) or B.1.617.3 Other variants |
76.1 (70-81.2) 88.3 (66.8-97) 75.2 (55.3-87) 71.9 (60.1 - 80.5) |
| 8 | Dunkle et al32 (2021 | USA & Mexico | NVX-CoV2373 (Novavax) | Protein Subunit | 19 714 | 9 868 | All types | 90.4 (82.9-94.6) |
| 9 | Ella et al26 (2021) | India |
BBV152 (Covaxin) |
Whole virus inactivated | 12 221 | 12 198 |
All types B.1.617.2 (delta) B.1.617.1 (kappa) Other variants |
70.8 (50-83·8) 65.2 (33·1-83) 90.1 (30·4-99·8) 73 (–2·2 - 95·2) |
| 10 | Emary et al23 (2021) | UK |
AZD1222 (Covishield) |
Vector | 4244 | 4290 |
B.1.1.7 All types |
77.3 (65·4-85·0) 61.7 (36·7-76·9) |
| 11 | Falsey et al24 (2021) | USA, Chile, & Peru |
AZD1222 (Covishield) |
Vector | 21,635 | 10,816 | All types | 74 (65.3-80.5) |
| 12 | Frenck et al28 (2021) | USA, Argentina, Brazil, Germany, Turkey, & south Africa |
BNT162b2 (Pfizer-BioNTech/Comirnaty) |
mRNA | 1131 | 1129 | All types | 100 (75.3-100) |
| 13 | Hager et al40 (2022) | Argentina, Brazil, Canada, Mexico, the United Kingdom, & the United States |
CoVLP+AS03 (Covifenz) |
Protein Subunit | 12 074 | 12 067 |
All types Alpha Gamma Delta |
69.5 (56.7-78.8) 100 (38.2 - NA) 87.8 (73 - 95.3) 74 (51.7 - 86.8) |
| 14 | Halperin et al39 (2021) | Argentina, Chile, Mexico, Pakistan, & Russia |
Ad5-nCoV (Convidecia) |
Vector | 18489 | 18 493 | All types | 57.5 (39·7-70) |
| 15 | Heath et al33 (2021) | UK |
NVX-CoV2373 (Novavax) |
Protein subunit | 7593 | 7594 |
All types B.1.1.7 Non-B.1.1.7 |
89.7 (80.2-94.6) 86.3 (71.3 to 93.5) 96.4 (73.8 to 99.5) |
| 16 | Kremsner et al27 (2021) | Europe & Latin America |
CVnCoV (CVnCoV) |
mRNA | 19846 | 19834 |
All types Alpha (B.1.1.7/501Y.V2) Gamma (P.1/501Y.V3) Lambda (C.37) |
48.2 (31–61·4) 55.1 (23·5–73·6) 67.1 (29·8–84·6) 52.8 (8·2–75·8) |
| 17 | Logunov et al43 (2021) | Russia |
rAd5 (Sputnik-V) |
Vector | 16501 | 5476 | All types | 91.6 (85·6–95·2) |
| 18 | Toledo-Romani et al42 (2022) | Cuba |
SOBERANA-02 (Soberna) |
Protein Subunit | 14679 | 14675 | All types | 69.7 (56.5-78.9) |
| 19 | Polack et al29 (2020) | USA & Germany |
BNT162b2 (Pfizer-BioNTech/Comirnaty) |
mRNA | 21,720 | 21,728 | All types | 95 (90.3-97.6) |
| 20 | Sadoff et al21 (2022) | USA, South Africa, Brazil, Colombia, Argentina, Peru, Chile, Mexico |
Ad26.COV2. S (Jcovden) |
Vector | 19,577 | 19,608 |
Overall B.1.1.7 (alpha) B.1.351 (beta B.1.617.2/AY.1/AY.2 (delta) B.1.427/429 (epsilon) P.1 (gamma) C.37 (lambda) P.2 (zeta) B.1.621 (mu) B.1.1.519 Other+E484K |
52.9 (47.1-58.1) 70.2 (35.3-87.6) 51.9 (19.1-72.2) -5.7 (−177.7-59.2) 65.7 (−0.9-90.3) 36.5 (14.1-53.3) 10.1 (−39.2 to 42.1) 64.1 (42.5 to 78.3) 35.9 (1.7 to 58.7) 51.9 (−235.4 to 95.7) 68 (26.3 to 87.6) |
| 21 | Sahly et al37 (2021) | USA |
mRNA-1273 (Spikevax) |
mRNA | 15,209 | 15,206 | All types | 93.2 (91-94.8) |
| 22 | Shinde et al45 (2021) | South Africa |
NVX-CoV2373 (Novavax) |
Protein Subunit | 2199 | 2188 | All types | 49.4 (6.1-72.8) |
| 23 | Tanriove et al41 (2021) | Turkey |
CoronaVac (Sinovac) |
Whole virus inactivated | 6650 | 3568 | All types | 83.5 (65·4–92·1) |
| 24 | Thomas et al31 (2021) | USA, Argentina, Brazil, South Africa, Germany, & Turkey |
BNT162b2 (Pfizer-BioNTech/Comirnaty) |
mRNA | 22,085 | 22,080 |
All types B.1.351 (beta) |
91.3 (89-93.2) 100 (53.5-100) |
| 25 | Voysey et al25 (2021) | South Africa, UK, & Brazil | AZD1222 (Covishield) | Vector | 8597 | 8581 | All types | 62.1 (41-75.7) |
| 26 | Walter et al30 (2021) | UK, Brazil, South Africa |
BNT162b2 (Pfizer-BioNTech/ Comirnaty) |
mRNA | 1528 | 757 | All types | 90.7 (67.7-98.3) |
The characteristics table displaying the characteristics of 26 included articles in the current network meta-analysis. VE, vaccine efficacy; CI, confidence interval
Quality of studies
All studies in the network meta-analysis have shown JADAD scores greater than or equal to 3 (threshold). Hence, all the RCTs were in the high domain in terms of quality i.e., low risk of bias. Supplementary table I presents the result of quality in detail.
Results of NMA
Our network meta-analysis of 59 studies unveiled the efficacy of 16 treatments, including 15 vaccines and one placebo. Surprisingly, heterogeneity was minimal, with statistical insignificance (Q-statistics =40.4, P=0.62) and an I2 of 0 per cent [95% confidence interval (CI) 0% to 34.5%]. This allowed us to confidently employ a fixed-effect model to reveal the pooled vaccine efficacy across comparisons. The network diagram (Fig. 2) showcases the diverse vaccines under scrutiny and their interconnected efficacy networks.
- The network diagram of treatments (vaccines and placebo) in the investigation, shows direct and indirect comparisons. The nodes in the network plot represent the vaccines plus the placebo, which are involved in the current network meta-analysis. The solid lines represent the direct connection between the vaccines under interest. As observed, the size of the solid lines connecting placebo with other vaccines, are directly proportional to the number of studies between them. Also, the size of the nodes represents the number of studies reporting the vaccines in network meta-analysis. The dotted lines represent the indirect comparisons between the vaccines under investigation, resulting in indirect estimates.
The direct estimates, with placebo serving as the reference, revealed significant vaccine efficacies as follows: JCovden [VE: 52.42 (47.28-57.57); P<0.05], Convidecia [VE: 57.50 (39.70-75.30); P<0.05], CVnCov [VE: 63.70 (52.20-75.20); P<0.05], ZyCoV-D [VE: 66.70 (47.60-85.60); P<0.05], Soberna [VE: 69.70 (56.50-82.90); P<0.05], Covaxin [VE: 70.81 (65.33-76.29); P<0.05], S-Trimer [VE: 71.61 (56.23-86.98); P<0.05], Covishiled [VE: 72.34 (67.12-77.56); P <0.05], Zifivax [VE: 75.94 (70.86-81.02); P<0.05], Covifenz [VE: 77.27 (68.48-86.06); P<0.05], Sinovac [VE: 83.50 (65.40-101.60); P<0.05], Novavax [VE: 88.99 (83.55-94.42); P<0.05], Sputnik [VE: 91.60 (85.60-97.60); P<0.0)], Comirnaty [VE: 92.07 (90.03-94.12); P <0.05], and Spikevax [VE: 93.29 (91.31-95.27); P<0.05]. The forest plot (Fig. 3A) intricately showcases the outcomes of the respective network meta-analysis. The result is displayed in Supplementary table II.
![The forest plots by taking (A) placebo, (B) Convidecia (C) Covaxin, (D) Covifenz, (E) Covishield, and (F) CVnCoV as reference treatments. The direct estimate was obtained by selecting (A) placebo as reference, all vaccines were significantly effective. Significant indirect estimates were obtained when (B) Convidencia [(positive significant vaccines: Sinovac, Novavax, Sputnik, Pfizer-BioNTech, and Spikevax) (negative significant vaccines: Placebo)], (C) Covaxin [(positive significant vaccines: Novavax, Sputnik, Pfizer-BioNTech, and Spikevax) (negative significant efficacy: JCovden, and placebo)], (D) Covifenz [(positive significant vaccines: Novavax, Sputnik, Pfizer-BioNTech, and Spikevax) (negative significant efficacy: JCovden, and placebo)], (E) Covishield [(positive significant vaccines: Novavax, Sputnik, Pfizer-BioNTech, and Spikevax) (negative significant efficacy: JCovden, and placebo)], and (F) CVnCov [(positive significant vaccines: Novavax, Sputnik, Pfizer-BioNTech, and Spikevax) (negative significant efficacy: placebo)].](/content/175/2025/161/1/img/IJMR-161-1-9-g3.png)
- The forest plots by taking (A) placebo, (B) Convidecia (C) Covaxin, (D) Covifenz, (E) Covishield, and (F) CVnCoV as reference treatments. The direct estimate was obtained by selecting (A) placebo as reference, all vaccines were significantly effective. Significant indirect estimates were obtained when (B) Convidencia [(positive significant vaccines: Sinovac, Novavax, Sputnik, Pfizer-BioNTech, and Spikevax) (negative significant vaccines: Placebo)], (C) Covaxin [(positive significant vaccines: Novavax, Sputnik, Pfizer-BioNTech, and Spikevax) (negative significant efficacy: JCovden, and placebo)], (D) Covifenz [(positive significant vaccines: Novavax, Sputnik, Pfizer-BioNTech, and Spikevax) (negative significant efficacy: JCovden, and placebo)], (E) Covishield [(positive significant vaccines: Novavax, Sputnik, Pfizer-BioNTech, and Spikevax) (negative significant efficacy: JCovden, and placebo)], and (F) CVnCov [(positive significant vaccines: Novavax, Sputnik, Pfizer-BioNTech, and Spikevax) (negative significant efficacy: placebo)].
The ranking of vaccines (in descending order) according to their efficacy based on P-score was Spikevax (0.95), Comirnaty (0.89), Sputnik (0.88), Novavax (0.81), Sinovac (0.71), Covifenz (0.59), Zifivax (0.57), Covaxin (0.44), S-Trimer (0.44), Covishield (0.42), Soberna (0.39), ZyCoV-D (0.33), CVnCov (0,25), Covindecia (0.18), and Jcovden (0.09).
For indirect estimates, first, Covidecia vaccine was selected as a reference vaccine, the significance results were obtained by placebo [VE: -57.50 (-75.30, -39.70); P<0.05], Sinovac [VE: 26.00 (0.61, 51.39); P = 0.04], Novavax [VE: 31.49 (12.87, 50.10); P<0.05], Sputnik [VE: 34.10 (15.32, 52.88); P<0.05], Comirnaty [VE: 34.57 (16.66, 52.49); P <0.05], and Spikevax [VE: 35.79 (17.88, 53.70); P<0.05]. Second, Covaxin was selected as a reference vaccine, the significant results were obtained by placebo [VE: -70.81 (-76.29, -65.33); P<0.05], JCoveden [VE: -18.39 (-25.91, -10.87); P<0.05], Novavax [VE: 18.17 (10.46, 25.89); P<0.05], Sputnik [VE: 20.79 (12.66, 28.91); P<0.05], Comirnaty [VE: 21.26 (15.41, 27.11); P<0.05], and Spikevax [VE: 22.48 (16.65, 28.30); P<0.05]. Similarly, Covifenz, Covisheild, CVnCov, JCovden, Novavax, Comirnaty, Sinovac, Soberna, Spikevax, S-timer, Sputnik, Zifivax and ZyCoV-D were selected as reference vaccines one after the other and indirect estimates were obtained. Forest plots (Fig. 3B-F, 4A-F and 5A-D) display results of respective network meta-analyses.
![The forest plots by taking (A) JCovden, (B) Novavax, (C) Comirnaty, (D) Sinovac, (E) Soberna, and (F) Spikevax as reference treatments. Significant Indirect estimates were obtained when (A) Jcovden [(positive significant vaccines: Soberna, S-Trimer, Covaxin, Covishield, Zifivax, Covifenz, Sinovac, Novavax, Sputnik, Pfizer-BioNTech, Spikevax) (negative significant vaccines: placebo)], (B) Novavax [(positive significant vaccines: No vaccine) (negative significant efficacy: Jcovden, Convidecia, CVnCov, Zydus Cadila, Soberna, S-Trimer, Covaxin, Covishield, Zifivax, Covifenz, placebo)], (C) Pfizer-BioNTech [(positive significant vaccines: No Vaccines) (negative significant efficacy: Jcovden, Convidecia, CVncov, Zydus Cadila, Soberna, S-Trimer, Covaxin, Covishield, Zifivax, Covofenz, placebo)], (D) Sinovac [(positive significant vaccines: No vaccines) (negative significant efficacy: Jcovden, Convidecia, placebo)], (E) Soberna [(positive significant vaccines: Novavax, Sputnik, Pfizer-BioNTech, Spikevax) (negative significant efficacy: Jcovden, placebo)], and (F) Spikevax [(positive significant vaccines: No vaccines) (negative significant efficacy: Jcovden, Convidecia, CVnCov, Zydus Cadila, Soberna, S-Trimer, Covaxin, Covishield, Zifivax, Covifenz)].](/content/175/2025/161/1/img/IJMR-161-1-9-g4.png)
- The forest plots by taking (A) JCovden, (B) Novavax, (C) Comirnaty, (D) Sinovac, (E) Soberna, and (F) Spikevax as reference treatments. Significant Indirect estimates were obtained when (A) Jcovden [(positive significant vaccines: Soberna, S-Trimer, Covaxin, Covishield, Zifivax, Covifenz, Sinovac, Novavax, Sputnik, Pfizer-BioNTech, Spikevax) (negative significant vaccines: placebo)], (B) Novavax [(positive significant vaccines: No vaccine) (negative significant efficacy: Jcovden, Convidecia, CVnCov, Zydus Cadila, Soberna, S-Trimer, Covaxin, Covishield, Zifivax, Covifenz, placebo)], (C) Pfizer-BioNTech [(positive significant vaccines: No Vaccines) (negative significant efficacy: Jcovden, Convidecia, CVncov, Zydus Cadila, Soberna, S-Trimer, Covaxin, Covishield, Zifivax, Covofenz, placebo)], (D) Sinovac [(positive significant vaccines: No vaccines) (negative significant efficacy: Jcovden, Convidecia, placebo)], (E) Soberna [(positive significant vaccines: Novavax, Sputnik, Pfizer-BioNTech, Spikevax) (negative significant efficacy: Jcovden, placebo)], and (F) Spikevax [(positive significant vaccines: No vaccines) (negative significant efficacy: Jcovden, Convidecia, CVnCov, Zydus Cadila, Soberna, S-Trimer, Covaxin, Covishield, Zifivax, Covifenz)].
![The forest plots by taking (A) S-Trimer, (B) Sputnik, (C) Zifivax and (D) ZyCoV-D as reference treatments. Significant Indirect estimates were obtained when (A) S-Trimer [(positive significant vaccines: Novavax, Sputnik, Pfizer-BioNTech, Spikevax) (negative significant vaccines: Jcovden, and placebo)], (B) Sputnik [(positive significant vaccines: no vaccine) (negative significant efficacy: Jcovden, Convidecia, CVnCov, Zydus Cadila, Soberna, S-Trimer, Covaxin, Covishield, Zifivax, Covifenz, placebo)], (C) Zifivax [(positive significant vaccines: Novavax, Sputnik, Pfizer-BioNTech, Spikevax) (negative significant vaccines: Jcovden, and placebo)], and (D) Zydus cadila [(positive significant vaccines: Novavax, Sputnik, Pfizer-BioNTech, Spikevax) (negative significant efficacy: placebo)].](/content/175/2025/161/1/img/IJMR-161-1-9-g5.png)
- The forest plots by taking (A) S-Trimer, (B) Sputnik, (C) Zifivax and (D) ZyCoV-D as reference treatments. Significant Indirect estimates were obtained when (A) S-Trimer [(positive significant vaccines: Novavax, Sputnik, Pfizer-BioNTech, Spikevax) (negative significant vaccines: Jcovden, and placebo)], (B) Sputnik [(positive significant vaccines: no vaccine) (negative significant efficacy: Jcovden, Convidecia, CVnCov, Zydus Cadila, Soberna, S-Trimer, Covaxin, Covishield, Zifivax, Covifenz, placebo)], (C) Zifivax [(positive significant vaccines: Novavax, Sputnik, Pfizer-BioNTech, Spikevax) (negative significant vaccines: Jcovden, and placebo)], and (D) Zydus cadila [(positive significant vaccines: Novavax, Sputnik, Pfizer-BioNTech, Spikevax) (negative significant efficacy: placebo)].
Publication bias
The Egger’s test showed a statistically insignificant (P= 0.85) result for publication bias. The funnel plot (Supplementary Figure) displayed closeness to symmetry (only one study out of an inverted funnel); hence, no publication bias was present.
Discussion
All vaccines demonstrated significant positive efficacy in the current investigation’s direct estimates. In indirect comparisons, Sinovac, Novavax, Sputnik, Comirnaty, and Spikevax outshone Convidecia with significant positive efficacy. However, JCovden displayed negative efficacy, while Novavax, Sputnik, Comirnaty, and Spikevax stood strong against Covaxin, Covifenz, and Covishield with positive efficacy. Against CVnCov, Novavax, Sputnik, Comirnaty, and Spikevax showcased notable positive efficacy. Soberna, S-Trimmer, Covaxin, Covishield, Zifivax, Covifenz, Sinovac, Novavax, Sputnik, Comirnaty, and Spikevax excelled when compared to JCovden; all with significant positive efficacy. Conversely, JCovden, Convidecia, CVnCov, ZyCoV-D, Soberna, S-Trimer, Covaxin, Covishield, Zifivax, and Covifenz displayed negative efficacy compared to Novavax and Comirnaty. Additionally, JCovden and Convidecia showed negative efficacy against Sinovac. The contrast continued as JCovden faltered while Novavax, Sputnik, Comirnaty, and Spikevax shone against Soberna. Moreover, JCovden, Convidecia, CVnCov, ZyCoV-D, Soberna, S-Trimer, Covaxin, Covishield, Zifivax, and Covifenz exhibited significant negative efficacy compared to Spikevax. JCovden also displayed negativity compared to both S-Trimer and Zifivax, while Novavax, Sputnik, Comirnaty, and Spikevax showed noteworthy negative efficacy against Zifivax and ZyCoV-D.
This investigation delves into the effectiveness of COVID-19 vaccines and offers crucial insights into their comparative efficacy. With limited data on head-to-head trials, our study presents significant findings on the positive and negative efficacy of various vaccines when pitted against each other. Representing a pioneering effort, this study presents the network meta-analysis encompassing 15 vaccines developed using four distinct technologies worldwide. Such comprehensive analysis lays a foundational framework for future vaccine development endeavours, highlighting the notable efficacy of mRNA vaccines against COVID-19.
Several previous studies with similar objectives made similar comparisons, though they differ methodologically from the current investigation. For instance, Taubasi et al46 (2022) incorporated placebo-controlled trials in their analysis, focusing on safety outcomes such as local, systemic, and unsolicited side effects. In contrast, the present investigation emphasises only vaccine efficacy as an outcome. Additionally, Taubasi et al46 (2022) did not specify inclusion criteria in a structured format like PICO, PICOS, or PICOT, which limits reproducibility. Furthermore, Wu et al47 (2024) applied a Bayesian network meta-analysis to their study, targeting a population aged >18 yr, as defined in their inclusion criteria. In comparison, the current investigation employs a frequentist network meta-analysis and includes a larger number of studies, potentially enhancing the robustness of the comparative findings.
The major limitation of the current investigation is all the vaccine’s efficacy during intercomparisons is obtained as indirect estimates. Hence, it is advisable to conduct sound methodological superiority or non-inferiority RCT to get robust estimates. It is difficult to comment on the duration of protection by COVID-19, as VE is estimated within a few days or months. The mutation properties of the virus should be accounted for before further investigation i.e., designing RCTs. The exposure-based study designs should also be considered to check that any variation in efficacy is due to the selection of different study designs. The period of primary RCTs is less and varies from study to study. In the current investigation, the vaccine efficacy is estimated irrespective of different kinds of COVID-19 variants. Hence, it is advisable to do a subgroup meta-analysis.
The vaccine’s efficacy should also be estimated by considering the re-infection rate from the vaccinated and non-vaccinated (natural immunity) groups. If natural immunity helps, as compared to the vaccinated groups, the efficacy of the vaccines becomes arguable.
Conclusion
We concluded that mRNA technology vaccines were more efficient against COVID-19 as compared to vaccines developed by other technologies (at least for a few days or months). Ranking of vaccines (in descending order) according to their efficacy was Spikevax, Comirnaty, Sputnik, Novavax, Sinovac, Covifenz, Zifivax, Covaxin, S-Trimer, Covishield, Soberna, ZyCoV-D, CVnCov, Covindecia, and Jcovden.
Financial support & sponsorship
None.
Conflicts of Interest
None.
Use of Artificial Intelligence (AI)-Assisted Technology for manuscript preparation
The authors confirm that there was no use of AI-assisted technology for assisting in the writing of the manuscript and no images were manipulated using AI.
References
- WHO Director-General’s opening remarks at the media briefing on COVID-19 - 25 May 2020. Available from: https://www.who.int/director-general/speeches/detail/who-director-general-s-opening-remarks-at-the-media-briefing-on-covid-19---25-may-2020, accessed October 18, 2023.
- A Novel Coronavirus from Patients with Pneumonia in China, 2019. N Engl J Med. 2020;382:727-33.
- [CrossRef] [PubMed] [PubMed Central] [Google Scholar]
- Efficacy, safety, and immunogenicity of the DNA SARS-CoV-2 vaccine (ZyCoV-D): The interim efficacy results of a phase 3, randomised, double-blind, placebo-controlled study in India. Lancet. 2022;399:1313-21.
- [CrossRef] [PubMed] [PubMed Central] [Google Scholar]
- COVID-19 vaccine tracker and landscape. Available from: https://www.who.int/publications/m/item/draft-landscape-of-covid-19-candidate-vaccines, accessed on October 18, 2023.
- mRNA vaccines - a new era in vaccinology. Nat Rev Drug Discov. 2018;17:261-79.
- [CrossRef] [PubMed] [PubMed Central] [Google Scholar]
- Fundamentals of vaccine immunology. J Glob Infect Dis. 2011;3:73-8.
- [CrossRef] [PubMed] [PubMed Central] [Google Scholar]
- Developing mRNA-vaccine technologies. RNA Biol. 2012;9:1319-30.
- [CrossRef] [PubMed] [PubMed Central] [Google Scholar]
- Viral vector vaccine development and application during the covid-19 pandemic. Microorganisms. 2022;10:1450.
- [CrossRef] [PubMed] [PubMed Central] [Google Scholar]
- Replicating viral vector-based vaccines for covid-19: Potential avenue in vaccination arena. Viruses. 2022;14:759.
- [CrossRef] [PubMed] [PubMed Central] [Google Scholar]
- A Comprehensive review of the protein subunit vaccines against covid-19. Front Microbiol. 2022;13:927306.
- [CrossRef] [PubMed] [PubMed Central] [Google Scholar]
- Chapter 5 - Vaccine delivery systems against tuberculosis. In: Kesharwani P, ed. Nanotechnology Based Approaches for Tuberculosis Treatment. Academic Press; 2020. p. :75-90.
- [Google Scholar]
- Effectiveness of whole-virus covid-19 vaccine among healthcare personnel, Lima, Peru. Emerg Infect Dis. 2022;28:S238-43.
- [CrossRef] [PubMed] [PubMed Central] [Google Scholar]
- How to read a network meta-analysis. Eur Urol Focus. 2023;9:701-4.
- [CrossRef] [PubMed] [Google Scholar]
- Network meta-analysis. Methods Mol Biol. 2022;2345:187-201.
- [CrossRef] [PubMed] [PubMed Central] [Google Scholar]
- The PRISMA extension statement for reporting of systematic reviews incorporating network meta-analyses of health care interventions: Checklist and explanations. Ann Intern Med. 2015;162:777-84.
- [CrossRef] [PubMed] [Google Scholar]
- PICO, PICOS and SPIDER: A comparison study of specificity and sensitivity in three search tools for qualitative systematic reviews. BMC Health Serv Res. 2014;14:579.
- [PubMed] [Google Scholar]
- On sample sizes to estimate the protective efficacy of a vaccine. Stat Med. 1988;7:1279-88.
- [CrossRef] [PubMed] [Google Scholar]
- Appendix: Jadad scale for reporting randomized controlled trials. In: Halpern SH, Douglas MJ, eds. Evidence-based Obstetric Anesthesia. Blackwell Publishing Ltd; 2005. p. :237-8.
- [Google Scholar]
- Cochrane handbook for systematic reviews of interventions. Available from: https://training.cochrane.org/handbook/current, accessed on August 17, 2023.
- Ranking treatments in frequentist network meta-analysis works without resampling methods. BMC Med Res Methodol. 2015;15:58.
- [CrossRef] [PubMed] [PubMed Central] [Google Scholar]
- Final analysis of efficacy and safety of single-dose Ad26.COV2.S. N Engl J Med. 2022;386:847-60.
- [CrossRef] [PubMed] [PubMed Central] [Google Scholar]
- Efficacy of ChAdOx1 nCoV-19 (AZD1222) vaccine against SARS-CoV-2 lineages circulating in Brazil. Nat Commun. 2021;12:5861.
- [CrossRef] [PubMed] [PubMed Central] [Google Scholar]
- Efficacy of ChAdOx1 nCoV-19 (AZD1222) vaccine against SARS-CoV-2 variant of concern 202012/01 (B.1.1.7): An exploratory analysis of a randomised controlled trial. Lancet. 2021;397:1351-62.
- [CrossRef] [PubMed] [PubMed Central] [Google Scholar]
- Phase 3 Safety and efficacy of AZD1222 (ChAdOx1 nCoV-19) covid-19 vaccine. N Engl J Med. 2021;385:2348-60.
- [CrossRef] [PubMed] [PubMed Central] [Google Scholar]
- Single-dose administration and the influence of the timing of the booster dose on immunogenicity and efficacy of ChAdOx1 nCoV-19 (AZD1222) vaccine: A pooled analysis of four randomised trials. Lancet. 2021;397:881-91.
- [CrossRef] [PubMed] [PubMed Central] [Google Scholar]
- Safety and immunogenicity of an inactivated SARS-CoV-2 vaccine, BBV152: A double-blind, randomised, phase 1 trial. Lancet Infect Dis. 2021;21:637-46.
- [CrossRef] [PubMed] [PubMed Central] [Google Scholar]
- Efficacy and safety of the CVnCoV SARS-CoV-2 mRNA vaccine candidate in ten countries in Europe and Latin America (HERALD): A randomised, observer-blinded, placebo-controlled, phase 2b/3 trial. Lancet Infect Dis. 2022;22:329-40.
- [CrossRef] [PubMed] [PubMed Central] [Google Scholar]
- Safety, immunogenicity, and efficacy of the BNT162b2 Covid-19 vaccine in adolescents. N Engl J Med. 2021;385:239-50.
- [CrossRef] [PubMed] [PubMed Central] [Google Scholar]
- Safety and efficacy of the BNT162b2 mRNA covid-19 vaccine. N Engl J Med. 2020;383:2603-15.
- [CrossRef] [PubMed] [PubMed Central] [Google Scholar]
- Evaluation of the BNT162b2 covid-19 vaccine in children 5 to 11 years of age. N Engl J Med. 2022;386:35-46.
- [CrossRef] [PubMed] [PubMed Central] [Google Scholar]
- Safety and efficacy of the BNT162b2 mRNA covid-19 vaccine through 6 months. N Engl J Med. 2021;385:1761-73.
- [CrossRef] [PubMed] [PubMed Central] [Google Scholar]
- Efficacy and safety of NVX-CoV2373 in adults in the United States and Mexico. N Engl J Med. 2022;386:531-43.
- [CrossRef] [PubMed] [PubMed Central] [Google Scholar]
- Safety and efficacy of NVX-CoV2373 Covid-19 Vaccine. N Engl J Med. 2021;385:1172-83.
- [CrossRef] [PubMed] [PubMed Central] [Google Scholar]
- Evaluation of mRNA-1273 SARS-CoV-2 vaccine in adolescents. N Engl J Med. 2021;385:2241-51.
- [CrossRef] [PubMed] [PubMed Central] [Google Scholar]
- Efficacy and safety of the mRNA-1273 SARS-CoV-2 vaccine. N Engl J Med. 2021;384:403-16.
- [CrossRef] [PubMed] [PubMed Central] [Google Scholar]
- Evaluation of mRNA-1273 covid-19 vaccine in children 6 to 11 years of age. N Engl J Med. 2022;386:2011-23.
- [CrossRef] [PubMed] [PubMed Central] [Google Scholar]
- Efficacy of the mRNA-1273 SARS-CoV-2 vaccine at completion of blinded phase. N Engl J Med. 2021;385:1774-85.
- [CrossRef] [PubMed] [PubMed Central] [Google Scholar]
- Efficacy and safety of the rbd-dimer-based covid-19 vaccine ZF2001 in adults. N Engl J Med. 2022;386:2097-111.
- [CrossRef] [PubMed] [PubMed Central] [Google Scholar]
- Final efficacy analysis, interim safety analysis, and immunogenicity of a single dose of recombinant novel coronavirus vaccine (adenovirus type 5 vector) in adults 18 years and older: An international, multicentre, randomised, double-blinded, placebo-controlled phase 3 trial. Lancet. 2022;399:237-48.
- [CrossRef] [PubMed] [PubMed Central] [Google Scholar]
- Efficacy and Safety of a recombinant plant-based adjuvanted covid-19 vaccine. N Engl J Med. 2022;386:2084-96.
- [CrossRef] [PubMed] [PubMed Central] [Google Scholar]
- Efficacy and safety of an inactivated whole-virion SARS-CoV-2 vaccine (CoronaVac): Interim results of a double-blind, randomised, placebo-controlled, phase 3 trial in Turkey. Lancet. 2021;398:213-22.
- [CrossRef] [PubMed] [PubMed Central] [Google Scholar]
- Safety and efficacy of the two doses conjugated protein-based SOBERANA-02 COVID-19 vaccine and of a heterologous three-dose combination with SOBERANA-Plus: A double-blind, randomised, placebo-controlled phase 3 clinical trial. Lancet Reg Health Am. 2023;18:100423.
- [CrossRef] [PubMed] [PubMed Central] [Google Scholar]
- Safety and efficacy of an rAd26 and rAd5 vector-based heterologous prime-boost COVID-19 vaccine: an interim analysis of a randomised controlled phase 3 trial in Russia. Lancet. 2021;397:671-81.
- [CrossRef] [PubMed] [PubMed Central] [Google Scholar]
- Efficacy of the adjuvanted subunit protein COVID-19 vaccine, SCB-2019: A phase 2 and 3 multicentre, double-blind, randomised, placebo-controlled trial. Lancet. 2022;399:461-72.
- [CrossRef] [PubMed] [PubMed Central] [Google Scholar]
- Efficacy of NVX-CoV2373 Covid-19 Vaccine against the B.1.351 Variant. N Engl J Med. 2021;384:1899-1909.
- [CrossRef] [PubMed] [PubMed Central] [Google Scholar]
- Efficacy and safety of COVID-19 vaccines: A network meta-analysis. J Evid Based Med. 2022;15:245-62.
- [CrossRef] [PubMed] [PubMed Central] [Google Scholar]
- Comparative efficacy and safety of COVID-19 vaccines in phase III trials: A network meta-analysis. BMC Infect Dis. 2024;24:234.
- [CrossRef] [PubMed] [PubMed Central] [Google Scholar]
