Procalcitonin-guided antibiotic usage - addressing heterogeneity in meta-analysis

Sir,

Shafiq et al1, in their meta-analysis, have evaluated the utility of procalcitonin (PCT)-guided antibiotic dosing for decision-making among inpatients in three different settings-intensive care units (ICUs), wards and emergency care. The outcomes of interest assessed were total mortality, 28-day mortality, need for ICU admission, proportion of patients treated with antibiotics, duration of antibiotic therapy and stay in hospital and antibiotic use per 1000 days of follow up. The authors concluded that while hard endpoints of mortality and need for ICU admissions remained unaffected; there was a significant reduction in the use of antibiotics in the emergency, ICU setting and the ward setting due to the use of PCT-guided dosing.

Meta-analyses such as these are important for driving policy particularly in resource-constrained settings. We, however, wish to raise some concerns regarding the methodological aspects of the meta-analysis and their impact on the authors' findings and conclusions.

The total number of studies included for the meta-analysis was 16 (n=8 studies in ICU, n=5 studies in the emergency setting and n=3 in the ward setting). None of the four figures depicting the Forest plots for the various outcome measures have all 16 studies included. For example, Fig. 2 (total mortality) shows only 12 studies, of which seven are in the ICU setting, four in the emergency setting and only one in the ward setting. In the ward setting, a single study is not actually amenable to a meta-analysis, and a minimum of two studies2 would be needed to draw any meaningful conclusion. Similarly, Fig. 5 (antibiotic use per 1000 days of follow up) has only two studies. This may be due to the fact that all 16 studies may not have reported all outcomes planned by the authors. Both the number of studies and their quality are important to the conclusions of a meta-analysis and the study by Shafiq et al1 is limited by small number of studies in both the emergency and ward settings. In addition, although the authors mention “low risk of bias” for the study quality, the methodology used to ascertain bias is not presented. Neither are all 16 studies listed in a Table 1 as per the requirement of the PRISMA statement3.

An important methodological issue with the analysis is the clinical heterogeneity among the studies included. The spectrum of disease included ranges from simple fevers to chronic obstructive pulmonary disease and bronchial asthma. In addition, the patients included in the study have ages ranging from less than a month to more than 65 years. This likely explains the heterogeneity seen in the various outcome measures (ranging from 0-65%). Two other meta-analyses45 on this subject had tighter inclusions and consequently lower heterogeneity (below 50%). Several methods (albeit with caveats) have been proposed to address heterogeneity and two of these include the use of the random effects model (a model that assumes that different studies have different true effects) and meta-regression (a regression technique that assesses whether the treatment effect is related to one or more characteristics of the studies or patients). The authors have used the random effects model for the outcomes of antibiotic usage (I² 90%), antibiotic use per 1000 days of follow up (I² 85%) and the proportion of patients treated with antibiotics (I² 81%). The fixed effects model has been used for total mortality (I² 0%). The study does not mention any rationale for the choice of either model. An additional challenge in any meta-analysis lies with the test for detecting heterogeneity which has low power when the sample sizes are small or when only a few trials are included. Both emergency and ward settings in the study by Shafiq et al1 have very small number of studies (2 or 3 studies only).

A major limitation of all meta-analyses worldwide in this area of PCT-guided treatment has been the presence of significant heterogeneity among included trials and diverse PCT guidance strategies (including antibiotic initiation, discontinuation or combination of antibiotic initiation and discontinuation strategies)6. Shafiq et al1 besides having significant heterogeneity in their meta-analysis, also acknowledge the use of differential protocols for PCT estimations.

The challenge of lumping and splitting studies for any meta-analysis is one that will continue to plague researchers in this area given that there is little or no guidance for authors7. While high heterogeneity itself should not preclude a meta-analysis, the rationale for the choice of the model used for the meta-analysis and the impact of the heterogeneity on the summary effect and the consequent conclusions need to be explicitly stated.