Evaluation of new non-invasive & conventional invasive methods of haemoglobin estimation in blood donors

Blood transfusion service is an integral part of the healthcare system throughout the world. Pre-donation check of haemoglobin (Hb) is a part of the standard operating procedure (SOP) for the blood donor selection. The conventional invasive technology to screen capillary Hb levels in blood donors is the semi-quantitative gravimetric copper sulphate (CuSO₄) method12. For estimation of Hb levels, the invasive quantitative point-of-care testing (POCT) such as HemoCue 301+ (HemoCue AB, Ängelholm, Sweden) is based on the principle of photometry and the non-invasive NBM200 (OrSense, Nes Ziona, Israel) is based on the principle of occlusion spectroscopy34. The non-invasive method has unique advantages of exemption of finger prick, the associated pain and breach in innate immunity barrier, possible exposure to medical staff and biomedical waste generation. Besides, unacceptable accuracy of the gravimetric semi-quantitative CuSO₄ method is being slowly replaced with more accurate POCT devices that provide quantitative Hb values with very short turn-around time. The validation of a new methodology is an important part of quality management system in blood transfusion services and must be done in the regional donor population.

Therefore, the present study was conducted to compare the invasive with the non-invasive methods of Hb estimation in blood donors in terms of classification functions [sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV), reliability, accuracy and concordance] and to assess the effect of independent donor variables of test results keeping the haematology analyzer (HA) as a reference method of Hb estimation.

Material & Methods

This cross-sectional study was conducted in the department of Transfusion Medicine, Postgraduate Institute of Medical Education & Research (PGIMER), Chandigarh, India, from June to September 2014 after obtaining clearance from the Institutional Ethics Committee. Consecutive blood donor who provided written informed consent were enrolled in the study (voluntary non-random).

The sample size for the study was calculated using the online software of Survey System (Creative Research Systems, USA). Taking the population into account, with expected margin of error of three per cent and 95 per cent confidence level, the sample size of the study was calculated to be 1100. The post hoc power of the study was found to be >95 per cent.

The study included blood donors selected or deferred in accordance with the Drugs and Cosmetics Act of 1940 and the Rules therein of 1945, Ministry of Health and Family Welfare (MoHFW), Government of India (GoI) as amended from time to time5.

The blood donors were selected or deferred on the basis of CuSO₄ method of Hb screening (Hb ≥12.5 g/dl) for the purpose of blood donation. For the study purpose, Hb values of the consenting participants (selected as well as deferred donors) were estimated by HemoCue and then by OrSense (on both side thumbs alternatively). A post-donation venous sample was drawn for analysis on HA from selected blood donors, based on the assumption that the time period of a blood donation was not sufficient enough to bring a change in the Hb value of the donor. In deferred donors, a venous sample was drawn only for the study purpose. Batch estimation of Hb levels of these samples was performed, within two hours from the time of sample collection, on the HA, the reference method (ORION 60, Ocean Medical Technology, New Delhi). Every day, manual external quality control was conducted with the reference control viz. Liquichek™ Hematology-16 Control (Bio-Rad Laboratories, USA). The working solution of CuSO₄ (specific gravity: 1.053 for a cut-off Hb ≥12.5 g/dl) was prepared from the stock solution and was released for use after quality control. HemoCue was calibrated by the service representative from the manufacturer. The participant's details such as age, gender, weight, pulse rate and blood pressure were recorded on the worksheet from the blood donor history questionnaire and consent form of the department adapted from the template drafted by the National Blood Transfusion Council, MoHFW, GoI6.

Statistical analysis: The statistical analyses were done by SPSS version 20.0 (IBM Corp., Chicago, USA) and Microsoft Excel 2010 (Microsoft Corp., WA, USA). Normality of data distribution was assessed by plotting box-plot charts. Data were presented as mean±standard deviation (SD). Mean variation was reflected by the bias which is the difference between the test and reference method. Classification functions (sensitivity, specificity, PPV, NPV and accuracy) of the test methods were assessed with reference to the HA. The donors were categorized into the following four groups viz. true positive (TP) donors who were truly deferred by screening methods (Hb <12.5 g/dl on both screening test and HA); false positive (FP) donors who were falsely deferred by screening methods (Hb <12.5 g/dl on screening test, but >12.5 g/dl on HA); true negative (TN) donors who were truly accepted by screening methods (Hb >12.5 g/dl on both screening test and HA); false negative (FN) donors who were falsely accepted by screening methods (Hb >12.5 g/dl on screening test, but <12.5 g/dl on HA). Reliability of the test methods was assessed by the intra-class correlation coefficient (ICC) (interpretation: <0: poor; 0.01-0.20: slight agreement; 0.21-0.40: fair agreement; 0.41-0.60: moderate agreement; 0.61-0.80: substantial agreement; and 0.81-1.00: almost perfect agreement)3. Bland-Altman analysis was performed to assess the concordance between the measurements obtained by the test and the reference methods. The limits of agreement (LoA) were calculated as mean difference ±2SD. Multivariable linear regression analysis was used to determine the effect of a set of variables independently on the bias. Receiver operating characteristic (ROC) curve was plotted for comparing the sensitivities of different Hb screening methods. All tests were two-sided with the level of significance fixed at 0.05.

Results

A total of 1100 participants were included; 18 were excluded due to technical errors in blood samples. Hb values of the remaining 1082 participants comprising 1051 selected and 31 deferred blood donors (Fig. 1)were taken into account for analysis. The study population consisted of 1071 males (99.16%) and 11 females (0.84%) (M:F ratio of 97:1); between 18 and 63 yr (30.17±7.97 yr) and weighing 47-117 kg (71.83±12.03 kg).

Close

Fig. 1

Comparative column chart showing the stratification of the donors based on haemoglobin values obtained by the different methods.

Export to PPT

Distribution of haemoglobin (Hb) data values: The Hb values obtained with HA and HemoCue showed normal distribution, while those with OrSense showed a negatively skewed distribution (Fig. 2).

Close

Fig. 2

Comparative Box-Whisker plot showing the distribution of the haemoglobin values as obtained by the haematology analyzer, the HemoCue and the OrSense.

Export to PPT

Mean haemoglobin values and bias: The mean Hb values on HA, OrSense and HemoCue and their respective mean deviations are depicted in Table I. The bias associated with OrSense was 0.89 g/dl [95% confidence interval (CI), −2.99, −1.21], whereas it was 1.05 g/dl (95% CI, −2.87, −0.77) with HemoCue (Table I).

Classification functions of test methods: The Hb values were stratified based on cut-off Hb value of ≥12.5 g/dl and are represented in a column chart (Fig. 1). The HA categorized 975 (90.11%) donors with Hb ≥12.5 g/dl and 107 (9.89%) donors with Hb <12.5 g/dl. The study participants were categorized as truly deferred, falsely accepted, truly accepted or falsely deferred based on the true value of Hb obtained on the HA (Table II).

The Hb values were further arbitrarily stratified as Hb of <8, 8-9.99, 10-11.99, 12-12.49, 12.5-16 and >16 g/dl based on HA to assess performance of test methods. All five participants with Hb <10 g/dl were correctly deferred by the CuSO₄, HemoCue and OrSense methods. Of the 48 participants with Hb between 10.0 and 12.0 g/dl, 35 (72.91%) were falsely accepted by CuSO₄, whereas 36 (75%) and 39 (81.25%) were falsely accepted by HemoCue and OrSense, respectively. Among the 54 participants with Hb between 12.0 and 12.49 g/dl, 52 (96.29%) donors were falsely accepted by CuSO₄, whereas 51 (94.44%) and 54 (100%) were falsely accepted by HemoCue and OrSense, respectively. Of the 926 participants with Hb between 12.5 and 16.0 g/dl, 11 (1.18%) donors were falsely deferred by CuSO₄, whereas four (0.43%) and nine (0.97%) were falsely deferred by HemoCue and OrSense, respectively. The remaining 49 participants with Hb >16 g/dl were correctly accepted by the CuSO₄, HemoCue and OrSense methods.

CuSO₄, HemoCue and OrSense demonstrated sensitivities of 18.7, 18.7 and 13.1 per cent and specificities of 98.9, 99.6 and 99.1 per cent, respectively. The PPVs were 64.5, 83.3 and 60.9 per cent; the NPVs were 91.7, 91.8 and 91.2 per cent and the accuracies were 90.9, 91.6 and 90.6 per cent, respectively.

Reliability of test methods: The ICC for OrSense was 0.726 (CI: 0.691, 0.756; P<0.001), demonstrating a substantial agreement, while that for the HemoCue was 0.851 (CI: 0.832, 0.867; P<0.001), demonstrating an almost perfect agreement Table I.

Concordance of test methods: Bland-Altman plots demonstrated a 2SD difference of >2.0 g/dl in Hb estimations between HA and HemoCue/OrSense. The deviation of the Hb values varied widely with HemoCue (0.774- −2.878 g/dl), as well as with OrSense (1.216- −2.999 g/dl) (Fig. 3A & B).

Close

Fig. 3

Bland-Altman plot of concordance for the different screening methods. (A) HemoCue and the haematology analyzer; (B) OrSense and the haematology analyzer; (C) OrSense (right) and the OrSense (left).

Export to PPT

Scatter plots of comparison of test methods: Scatter plots demonstrated a positive association between the two methods i.e. the values of Hb estimated by the HemoCue and the OrSense increased with the increase in the true value of Hb value (Fig. 4).

Close

Fig. 4

Scatter plot showing comparison between the haemoglobin values estimated by (A) OrSense with respect to haematology analyzer, and (B) HemoCue with respect to haematology analyzer.

Export to PPT

Effects of independent variables on haemoglobin estimation: Multivariable linear regression analysis showed that age, weight and pulse rate were associated with the bias for the HemoCue whereas gender and systolic blood pressure were associated with the bias for the OrSense. True value of Hb was associated with the bias observed with both HemoCue and OrSense (Table III).

Receiver operating characteristic (ROC) curves: The comparison of sensitivity and specificity for the screening methods of Hb estimation was done by plotting the ROC curves which showed higher sensitivity of the HemoCue as compared to the OrSense. The area under the curve (AUC) for the HemoCue was 0.890 (CI: 0.851, 0.930) and that for the OrSense was 0.808 (CI: 0.763, 0.854) (Fig. 5A).

Close

Fig. 5

Comparative receiver operating characteristic (ROC) curve for screening methods (A) OrSense (mean) (OSM) and HemoCue; (B) OrSense (left) (OSL) and OrSense (right) (OSR).

Export to PPT

Concordance of haemoglobin values obtained with OrSense: The bias between the two sets of Hb values as obtained from either of the thumbs was 0.035±0.6 g/dl. The Bland-Altman plot corroborated with the interpretation that the deviation in the Hb values, as estimated by OrSense, was not affected by the chosen thumb side (Fig. 3C). The ROC curves showed similar sensitivities irrespective of the side of the thumb used for the Hb estimation in the study. The AUC for the OrSense (right) was 0.811 (CI: 0.768, 0.855) and that for the OrSense (left) method was 0.786 (CI: 0.737, 0.835) (Fig. 5B).

Discussion

Data were normally distributed for Hb values on HA and HemoCue. OrSense demonstrated a left-skewed deviation (negative skewness) i.e. it would overestimate Hb in donors with low true Hb values (<10 g/dl) and underestimate Hb in donors with high Hb values (>16 g/dl). Kim et al3 have reported a left-skewed deviation with OrSense and slight right skewness with HemoCue; however, model of HemoCue (201+ or 301+) was not specified in their study. The mean deviations of Hb with the test devices and the reference method were within the acceptable error of ±1 g/dl as per the standards laid down by the International Council for Standardization in Haematology7.

Bias observed with HemoCue of 1.05 g/dl in the present study was comparable to 1.19 g/dl reported by Bhaskaram et al8 from India. Other studies reported bias for HemoCue ranging from 0.21 to 0.83 g/dl38910111213. OrSense demonstrated a bias of 0.89 g/dl which was higher compared to 0.66 g/dl reported by Singh et al12 from India. However, the other studies reported a much lower bias with OrSense ranging from 0.10 to 0.29 g/dl310111415.

CuSO₄ falsely accepted 87 of 1082 (8%) donors for the blood donation in the present study, which compared well with the five per cent reported by Malukani et al16. The false acceptance with HemoCue was 87 (8%), similar to that (6%) reported by Patel et al17. However, the false acceptance with OrSense was 93 (8.6%) which was higher than 1.5 and 2.5 per cent reported by Malukani et al16 and Ardin et al14 respectively. CuSO₄ falsely deferred 11 of 1082 (1%) donors, which was in contrast to 29 per cent reported by Sawant et al18; this variation could be due to reasons ranging from preparation, quality control to storage and the technical competence and working diligence that could vary from person to person. These reasons have been of concern during the use of the semi-quantitative methodology. HemoCue falsely deferred four donors (0.37%), which was lower than 3.3 per cent reported by Patel et al17. However, OrSense falsely deferred nine (0.8%) which was lower in comparison to three and 5.8 per cent reported by Malukani et al16 and Ardin et al14, respectively.

The sensitivity of test methods was similar for CuSO₄ (18.7%) and HemoCue (18.7%) but slightly lower for OrSense (13.1%). Other studies reported sensitivity for HemoCue ranging from 18.6 to 99.4 per cent and from 38.6 to 98 per cent for OrSense3101213181920. The specificity of test methods was similar for CuSO₄ (98.87%), OrSense (99.07%) and HemoCue (99.58%). The reported specificity for HemoCue ranges from 45 to 99.8 per cent and from 79.5 to 99.1 per cent for OrSense in other studies3101213181920 (Table IV).

The PPV of the screening test is important while deciding on the method of choice for Hb estimation in blood donors where focus is towards both blood donor safety and avoidance of unnecessary deferral of potential blood donors. The PPV of test methods was comparable for CuSO₄ and OrSense but significantly higher for HemoCue. The reported PPV ranges from 43 to 98.9 per cent for HemoCue and 24.8 to 30.2 per cent for OrSense in other studies12131820. As compared to other studies, HemoCue, being the screening test with highest PPV, may be the best available option in the invasive modalities. However, the non-invasive method viz. OrSense, with little less sensitivity and comparable specificity, when compared to HemoCue may be a viable alternative for donor screening with the advantage of exemption of both finger prick and biomedical waste generation.

NPV is important for donor safety. The NPV of test methods was comparable for CuSO₄, HemoCue and OrSense. The reported NPV for HemoCue ranges from 57.9 to 99 per cent and for OrSense from 94.3 to 95.8 per cent in other studies12131820. As compared to other studies, the OrSense demonstrated an NPV which was as good as the CuSO₄ and the HemoCue in the present study. The accuracy of each of the test methods was comparable. Ziemann et al20 have reported accuracy for HemoCue of 98.7 per cent.

Age, weight, pulse rate and true value of Hb were found to be associated with the bias for the Hb estimation by the HemoCue method, whereas gender, systolic blood pressure and true values of Hb were associated with the bias with OrSense. Gayat et al15 have reported that gender, heart rate, temperature, perfusion index and true value of Hb affect the bias associated with the OrSense. This difference in observation may be attributed to the ethnic differences and differences in the characteristics of the study population as well12.

The study had certain limitation as it was conducted in a centrally air-conditioned set-up with minimal variation in ambient temperature and humidity. Hence, the effects of the change in ambient temperature, humidity and dust in outdoor blood collection setups, on the measurements of Hb by these methods could not be assessed. The perfusion index which affects the Hb estimation by the OrSense was not taken into account in this study15. Whole blood donation is known to lower a donor's Hb level by approximately 0.5 g/dl immediately afterwards was not taken into account while estimating the Hb values by the HA14. The small number of female donors enrolled in the study was a limitation for assessing the effect of gender on the bias associated with the Hb measurements.

In conclusion, the present finding showed that non-invasive modality could provide the near-ideal pre-donation Hb screening platform only if improvement in sensitivity and PPV was possible keeping in view the unique advantage of exemption of both finger prick and biomedical waste generation.