Drug utilisation patterns & clinical outcomes in hospitalised COVID-19 patients: A geospatial & machine learning approach

Study design and population

Despite extensive global research on COVID-19, gaps persist in understanding between drug utilisation, clinical outcomes, and spatial mortality patterns, especially in under-represented regions like Sikkim. Most studies focus on isolated aspects, such as drug efficacy or mortality risk factors, without integrating machine learning and geospatial techniques. This retrospective cross-sectional chart review was conducted in a tertiary care teaching hospital in Sikkim to analyse the drug utilisation practices and adherence to standard treatment guidelines in the management of hospitalised, laboratory-confirmed SARS-CoV-2 positive cases. The study also aimed to identify models that could offer insights into the associations between specific medications, age, gender, comorbidity, and patient outcomes as clinical endpoints (death or discharge).

Inclusion/exclusion criteria

All SARS-Cov-2-positive patients of all genders aged ≥ 18 yr belonging to all ethnic groups admitted to the designated COVID ward of the hospital were included in the study. However, patients under 18 yr and admitted under obstetric case were excluded from the study. The exclusion of individuals under 18 yr was done primarily based on significant differences in the clinical presentation, disease progression, and the differences in treatment protocols of COVID-19 in children and younger age groups. All patients admitted in-between April 26, 2021 to June 26, 2021 (complete enumeration for the period of 2 months) were included in the study. This helped ensure comprehensive data collection, given the small population and limited case numbers and to analyse the drug utilisation patterns without the need for sampling, reducing the risk of selection bias and enhancing the representativeness of our findings.

Study timeline and methodological phases

The timeline for the study was systematically structured to ensure all the analysis, from data preprocessing to model validation and geospatial visualisation. Data acquisition, preprocessing, and cleaning were completed within four wk, including handling null values and encoding categorical variables. Data analyses were conducted over two wk to identify patterns and relationships. The machine learning workflow, model training, hyperparameter tuning, and testing spanned five wk. This process involved iterative adjustments to identify the best-performing models for regression and classification tasks. Geospatial analyses required two weeks, including the development of heatmaps and computation of Moran’s I index. The study was completed within a carefully planned 15-wk period, ensuring comprehensive analysis and validation of findings.

Data cleaning and bias mitigation

The data obtained included demographics (age, gender, ethnicity), length of hospital stay (admission date/discharge date), medications used, and clinical endpoints (death/discharge). The hospital data did not contain the latitude and longitude data, which was then added through the Google API using the address provided in the hospital data. The paper-based records were manually typed and converted into a digital format and were stored in an MS Excel file. The data was then pre-processed and prepared for regression and classification analysis. Initially, null values were identified and removed from the dataset. A total of 380 patients’ data qualified to be included in the study and were used for analysis. Comma-separated values occurred when multiple values were entered in a single field, such as medicine used or symptoms, and were separated into individual columns. Categorical variables were encoded using label encoding. In label encoding each unique category or label in a categorical variable was assigned a unique integer value.

Dataset bias posed a potential threat to validation in this study, as the data were exclusively sourced from a single tertiary care hospital in Sikkim, limiting demographic and clinical diversity. To address this, we performed rigorous preprocessing to handle missing values and ensure uniform encoding of categorical variables. Balanced sampling was applied during model training to mitigate class imbalance, particularly for clinical endpoints like mortality. K-fold cross-validation validated model performance across data splits, reducing overfitting risks. Feature importance analysis via the Random Forest model minimised the impact of disproportionately weighted variables. These measures improved the reliability and generalisability of the findings, though future multi-centre studies are needed for broader validation.

Drug utilisation and clinical endpoints

All medications given to the SARS-CoV-2 patients during their hospitalisation were collected and recorded. Medications received by the patients were further classified according to the Anatomical Therapeutic Classification (ATC-I, II, and V) system shown in table I. In this table, we presented a sample of medicine classification data, illustrating the categorisation of a portion of the total medications utilised. The complete list of remaining medicines has been provided in the supplementary file.

Three categories were used, as follows, regarding medication details⁴:

• (i)

  Medicines repurposed to treat COVID-19 (these included pharmacological agents, including remdesivir, azithromycin, and ivermectin that were being studied or reported to have possible effects against COVID-19).

• (ii)

  Supportive drugs (these included supportive treatment recommended for the treatment of COVID-19 patients, such as blood thinners, corticosteroids, and antibiotics).

• (iii)

  Other drugs (these included drugs, which might be useful for COVID-19 patients, such as statins, omeprazole, montelukast, cetirizine, etc.).

The evaluation of drug use started on the day of admission and went on until the clinical endpoints of interest were achieved – inpatient mortality or hospital discharge – occurred. The data were analysed to determine the relationship between age, gender, and medication use and the achievement of the clinical endpoint (discharge/death) utilising various mathematical models. Medication used, age, and gender were considered as independent variables, whereas attainment of the clinical endpoint was taken as dependent variable. The dataset was split into two subsets: a training set used to train the models, and a testing set to evaluate their performance. We used 80 per cent of the data for training purposes and 20 per cent data for testing purposes. We used an 80/20 split to maintain consistency with standard practices in machine learning while ensuring enough data for model training to learn the underlying patterns effectively. random forest (RF), multi-layer perceptron (MLP), ridge regression and logistic regression algorithms were used to analyse the data.

Regression and classification analysis

Regression and classification analysis are two important supervised machine-learning techniques used to analyse data and make predictions. The regression model was used to examine the relationship between a dependent variable and one or more independent variables. This model predicted the value of the dependent variable based on the values of the independent variables. Classification analysis was used when the predictive variable was categorical or discrete, with a limited number of classes. In the current study, we performed comparison analyses on regression and classification algorithms. Comparison analysis of machine learning models provides a best-fit algorithm with the data. These steps ensured the reliability and accuracy of our predictions regarding the relationship between drug utilisation and clinical outcomes. This comparison helped us validate the robustness of the analyses. For regression analysis, we used RF, MLP, and Ridge Regression. In classification analysis, we used logistic regression, RF, and MLP. Mean Squared Error, R² value is treated as a quantitative parameter to evaluate and compare the effectiveness of regression algorithms. Accuracy and Area Under the Receiver Operating Characteristic Curve (AUROC) scores validated the classification model. A random forest model was used to identify feature importance values. Feature importance values represent the contribution of each feature to the predictive power of the model for the binary classification task. Feature importance values are usually relative and can be interpreted as the impact of each feature on the model’s ability to discriminate between two classes. The feature importance values generated by the model provide insights into the relative contribution of each variable to the model’s predictive accuracy. Feature importance considers the combined and interactive effects of all variables to generate the values.

This study employed machine learning models rather than traditional statistical hypothesis testing to analyse drug utilisation patterns and predict clinical outcomes. The analysis was conducted using Python (Scikit-learn, NumPy, SciPy) for machine learning and ArcGIS Pro for geospatial analysis. Model performance was assessed using accuracy, AUROC, Mean Squared Error (MSE), and R² values, allowing for a comprehensive evaluation of classification and regression tasks. Feature importance was determined using the RF model to identify key predictors of clinical outcomes. The Moran’s I index and z-scores were used to analyse geospatial clustering of mortality. These methods provided an objective, data-driven approach for evaluating drug utilisation and patient outcomes without relying on conventional statistical significance tests. The software used in this study was Anaconda (2023.09-0) for machine analysis and ArcGIS Pro (3.1.4) for Geospatial analysis.

In regression models, the average squared difference between the predicted and actual values in a regression model is quantified by the MSE.

The lower values of the MSE represent a closer fit between expected and actual outcomes and thus serve as a measure of each model’s prediction accuracy¹⁵. R-squared (R²) quantifies the percentage of the dependent variable’s variation that can be accounted for by the independent variables.

An R² value equal to 1 indicates a perfect fit, whereas 0 indicates that the model is unable to explain any variation¹⁶. Negative values imply that the model is worse than the simple mean.

In classification models, accuracy determines the proportion of correctly classified instances out of the total instances in a classification model. Higher accuracy suggests better overall performance.

AUROC evaluates the area under the ROC curve, representing the trade-off between true positive rate (sensitivity) and false positive rate (1 - specificity)¹⁷.

AUROC = 0.5 implies no discrimination, while higher values signify better performance. An AUROC of 1 indicates a perfect classifier.

Moran’s I index

Moran’s I index is a measure used in spatial statistics to assess the spatial autocorrelation of a dataset, indicating the degree of similarity between nearby observations. It quantifies whether similar values tend to be clustered together or dispersed across a geographic area. The formula for Moran’s I involves the summation of products of the differences between each pair of values, the spatial weights matrix, and the mean value of the entire dataset. We have also used spatial autocorrelation in our dataset using Moran’s I index¹⁸ to identify if the data is clustered or dispersed.

The equation for Moran’s I is:

Here, n represents the number of spatial units or observations $x_i$ and $x_j$ are the values of the variable of interest at locations i and j, $\overline{x}$ is the mean of all observations, and $w_{ij}$ denotes the spatial weight between locations i and j. The index ranges from -1 (perfect dispersion) to 1 (perfect clustering), with 0 indicating a random spatial pattern.

Algorithm performance

We performed comparison analysis on three regression and classification algorithms. Our objective was to establish correlations between the medications administered to patients, their age, and gender, in relation to the endpoint, which signifies whether a patient’s outcome resulted in ‘Death’ or ‘Discharge’. This study compared the performance of several machine learning models for classification and regression tasks as shown in table II. For classification, RF, Logistic Regression, and MLP models were evaluated on their ability to predict categorical outcome variables. RF performed the best with an accuracy of 90.7 per cent and AUROC of 0.86, outperforming the other models. For regression problems, RF, MLP, and Ridge Regression models were tested. RF yielded the lowest MSE of 0.080 and highest R-squared of 0.144, indicating it best captured the variability in continuous target variables compared to the other regression models.

Overall, RF emerged as the top performer across classification and regression modelling. It achieved the highest accuracy and AUROC for classification, as well as the lowest MSE and highest R-squared in regression problems. These results suggest RF to be a robust machine learning approach that effectively handles both categorical and continuous target variables. It outperformed other commonly used algorithms like Logistic Regression, MLP, and Ridge Regression on this dataset.

Visualisations

As per the ATC medicine classification, we classified our data as shown in table I. Top 20 medicines used by the patients during their treatment were identified and demonstrated in figure 1. Similarly, the total patient’s clinical severity was determined, and gender-wise categorisation was done, as shown in table III. The total value in table III indicates the total number of patients. Gender-wise numbering was done based on severity category. As per our visualisation in table III, male patients had the highest percentage of death. Also, in the case of comorbidity, stroke, lung diseases, and sepsis had the highest death percentage.

Close

Fig. 1.

This figure shows the top 20 highest-used medicines to the study participants during the treatment.

Export to PPT

The feature importance values identified using the RF model are shown in table IV. The higher feature importance values indicate that a particular feature has a stronger influence on the model’s prediction. In table IV, we have shown the feature importance values from our best-performing model (RF classification model). Table IV shows that methylprednisolone had the highest feature importance value of 0.018907, indicating it was the most influential predictor of outcome. Visualising table IV helped us to understand which medicines were found to be most impactful for predicting whether a patient was discharged or died. Methylprednisolone emerged as the strongest individual predictor of increased mortality risk. The age-wise distribution of death patients shown in figure 2 reveals that mortality was highest among older age groups. The highest number of deaths (7 patients) occurred in the 61-70 yr age group, followed by six deaths in the age bracket 71-80 yr. Only three deaths were reported below the age of 30.

Close

Fig. 2.

Age-wise distribution of deceased patients.

Export to PPT

GIS analysis

GIS stands for Geographic Information System and the technology combines computer hardware, software, data, and analytical methods to capture, manage, analyse, and present spatial and geographic data. We used our data to visualise the heatmap for expired male and female patients. The heatmaps shown in supplementary figure 1 and 2 present the area with the highest number of expired female and male patients. The higher colour intensity in the map means increased density of patients and lower intensity colour means indicates sparse patient density. We can visualise in supplementary figure 1 that female patients from the East district of Sikkim had the highest number of death cases. In the case of male patients, as shown in supplementary figure 2, we can visualise that East district patients, and a lower number of North district study participants had expired. The observed clustering may reflect gender-specific vulnerabilities influenced by healthcare access, comorbidities, or differences in disease progression. These insights suggest the need for targeted interventions, such as improving healthcare infrastructure and outreach programmes, particularly in the East district for females and in the North district for males. The clustering underscores the importance of investigating underlying socio-economic and healthcare disparities that may disproportionately affect certain gender groups in these regions. We also visualised a Spatial Autocorrelation in our data using Moran’s Index through ArcGIS Pro. We achieved a Moran’s Index value of 0.125080 and a z-score value of 8.642819. These values indicate a clustered pattern in our data with a higher significance level.