Some newer & simpler biostatistical approaches for more credible clinical research

Abhaya Indrayan; Shivani Saini

doi:10.25259/IJMR_457_2025

View/Download PDF

Buy Reprints

PDF

Translate this page into:

Perspective

ARTICLE IN PRESS

doi:

10.25259/IJMR_457_2025

Some newer & simpler biostatistical approaches for more credible clinical research

Abhaya Indrayan^,*, Shivani Saini

Department of Clinical Research, Max Healthcare New Delhi, India

* For correspondence: a.indrayan@gmail.com

Received: 2025-02-19, Accepted: 2025-06-02, Epub ahead of print: 2025-10-24,

Licence

This is an open-access article distributed under the terms of the Creative Commons Attribution-Non Commercial-Share Alike 4.0 License, which allows others to remix, transform, and build upon the work non-commercially, as long as the author is credited and the new creations are licensed under the identical terms.

Biostatistical validation of the results is many times considered essential for their credibility. Without this, the results are suspected and rarely published; yet, the results with claimed validity frequently fail to produce the expected outcome in real-world applications¹. Statistical fallacies in clinical research can jeopardize life and health of many and need to be addressed more seriously than currently done².

Among the reasons for discrepancy between published research and practical applications, one important factor could be the use of inadequate methods of statistical evaluation, leading to incomplete or erroneous conclusion. Newer approaches are now available that can increase the applicability of the medical research results. We describe some relatively unknown new approaches in this communication that have wide applicability and are helpful in conducting more credible clinical research. A summary is provided at the end.

Assess medical significance besides statistical significance

Cohen³ has discussed uses and misuses of P values. Extensive discussions have taken place on statistical significance, culminating in a passionate plea to go beyond P<0.05⁴. However, the medical significance of the results has not received similar attention despite the use of this term at least since 1945 in a clinical context⁵.

The P value measures the likelihood of the results not inconsistent with the null hypothesis. This value depends on many, largely uncontrollable parameters such as the standard deviation (SD) and the effect size, but also the sample size, which is in our control. It is well known that a trivial effect becomes statistically significant with a small P value when the sample size is large⁶. However, a P value neither represents the importance of the results nor represents the effect size⁷. Thus, statistical significance by itself is losing significance, and we are moving to the medical significance of the results⁸.

Clinical research is more useful when the target is a sizeable effect that will justify adopting a new regimen. For instance, not many clinicians may like to abandon the well-tested existing regimen in favour of the new regimen for a minor gain of 1-2 per cent in efficacy, even when statistically significant. A substantial size of the effect that changes the current medical practice and abandons the existing one is called the medically significant effect. Thus, the effect size deserves more importance than currently given in most clinical research studies. The null hypothesis that there is no effect may soon vanish from the literature, even P values have a risk of going into obscurity unless some modifications are done.

A pragmatic approach is to pre-fix a target effect size considered medically significant and check that the results achieve statistical significance with respect to that effect size. The difficulty, however, could be to reach a consensus on what threshold of effect size is to be considered medically significant. Besides the variation in perception across clinicians, it would most likely differ from situation to situation. The researcher may fix a threshold with justification.

A pre-determined target for the detection of effect size has two disparate statistical implications that require newer approaches:

(i) The test of hypothesis

The null hypothesis under test in a new approach will be H₀: effect size ≤ δ against the alternative H₁: effect size > δ, where δ is the threshold of the effect size for medical significance for better clinical applicability. This will be a one-tailed test since the objective is to find whether the effect size is more than δ. The conventional test of significance for δ = 0 has little clinical relevance. For example, the observed mean reduction of 7 per cent in leak by nasal airway compared to conventional ventilation in cases undergoing bariatric surgery with SD = 6 per cent in a series of 64 cases would be statistically significant for H₀: mean reduction = 0. If the researcher decides that it should be at least δ = 20 per cent for switching to a nasal airway in these patients because it requires some extra efforts, the observed reduction of 7 per cent is not medically significant. The H₀: effect size ≤ δ is easy to test by computing Student-t as

$\frac{Observed effect size - δ}{Estimated SE of the effect size under H_{0}}$

This is particularly so for quantitative parameters, provided other conditions such as independence of the observations and Gaussian (Normal) distribution of the values are met. Gaussianity is almost assured for large samples because of the central limit theorem. A similar null can be set for other tests, such as chi-square. When the null effect size = δ is rejected against the alternative that is more, any effect size <δ is automatically rejected.

(ii) Sample size

The sample size formulae require specification of the effect size to be detected (or not to be missed) at a pre-fixed level of significance with specified power, where power is the probability of detecting that effect size when present. Detecting in our context means that this is found statistically significant. The square of the effect size (δ²) occurs in the denominator of all sample size formulae for testing of hypothesis situations. Some formulae appearing in the literature make the error of considering the observed effect size in a previous study as the size to be detected. This error was made by Kang⁹ for G*Power software and by Egbuchulem¹⁰ while discussing the basics of sample size determination. For comparing two proportions (one-tail), the formula generally used is $n \geq {\frac{[Z_{1 - α} \sqrt{2 \bar{p} (1 - \bar{p})} + Z_{1 - β} \sqrt{p_{0} (1 - p_{0}) + p_{1} (1 - p_{1})]}}{{(p_{1} - p_{0})}^{2}}}^{2}$ per group, with the usual notations. This is fallacious since the effect size to be detected must be the medically significant effect that has the potential to change the current medical practice and not the effect observed in a previous study. The denominator needs to be changed to a medically significant effect δ. For example, the correct formula for comparing two proportions (one-tail) is $n \geq {\frac{[Z_{1 - α} \sqrt{2 \bar{π} (1 - \bar{π})} + Z_{1 - β} \sqrt{π_{0} (1 - π_{0}) + π_{1} (1 - π_{1})]}}{δ^{2}}}^{2}$ per group where $π_{1} = π_{0} + / - δ$ depending on increase is anticipated or decrease. This is now being increasingly realized and adopted. This should not be confused with the sample size formula for equivalence trials, where the denominator is ${[| π_{1} - π_{0} | - δ]}^{2}$ . The correct formula has $π$ and not the sample estimate p (proportion).

Almost all sample size formulae require simple random sampling and Gaussian distribution of the estimate of the parameter involved in the calculations. Both tend to be ignored in practice. It is essential to make adjustments for sampling other than simple random and for distributions other than Gaussian. For sampling other than simple random, the sample size is typically adjusted by the design effect¹¹. For sampling from non-Gaussian distributions, the central limit theorem is invoked, and a large sample is advised. These adjustments are in addition to the well-known adjustments for expected non-response. Further adjustment is required for using sample estimate(s) for SD and p for proportion, whereas the calculation of sample size requires the population value of the parameter. Many popular texts, such as by Chow et al¹², wrongly use sample estimates in the sample size formulae instead of the correct population parameters. However, since the parameter value is not known, it is replaced by its estimate for practical purposes. This approximation requires adjustment but is almost never done (for example, see Egbuchulem¹⁰). We subjectively advise that the sample size be inflated by 10 per cent to account for this approximation.

Use P<0.01 for statistical significance and not P<0.05

A P value is calculated to measure the confidence that the study-data does not counter the null hypothesis. Even while testing the null H₀: effect size ≤ δ against that it is more, in fact for all tests statistical significance, there is a growing realization that P<0.01 should be considered for statistical significance in medical research in place of the prevalent P<0.05. Benjamin and Berger¹³ made a plea to use even P<0.005 for a novel discovery because people’s life and health are at stake. P<0.01 is a greater assurance that the required effect size is indeed present. Although the chance of missing the effect size when present (Type-II error) increases with P<0.01, that is not considered a big loss since it returns us to the status quo. This is temporary since the statistical significance will appear when repeat study with better methodology is done if the researcher is confident that the required effect is present.

Prefer individual comparisons over group comparisons

In a paired setup, such as the values of a liver function parameter before a treatment and after a treatment, the usual procedure is to use paired t-test for means of quantitative data, and McNemar test for counts in qualitative data. Whereas McNemar test is built on one-to-one matching, the paired t-test, despite being based on individual differences, compares the means. A small mean difference is likely to be adjudged not significant, even when some differences are extremely large on positive side and some on negative side. In a study on the blood characteristics pre- and post-chemotherapy in cancer patients¹⁴, if it was 10.38 in one case and improved to 14.05, the observation has great clinical significance, but the paired t-test would disregard this finding because it is based on mean difference. Large differences in individual subjects may contain cardinal information and can be flagged for further investigations that may suggest some new mechanisms causing such large differences in specific cases. This is rarely done. The new approach for paired data is to examine each difference and count how many of them agree within the clinical tolerance. For example, for a study on Hb level before and after a supplement for a month, perhaps a gain of less than 0.5 g/dL can be considered trivial and the proportion of cases with gain of 0.5 g/dL or more can be the right criterion to find the percentage of cases in whom the supplement was effective. This procedure is very simple and direct. Tests of hypothesis and confidence interval can be built for the percentage agreement (or disagreement, as needed) using binomial distribution even for small samples. Currently, this is rarely done.

The same kind of errors and other problems have been listed with Bland-Altman method and the method of one-to-one agreement within clinical tolerance has been proposed¹⁵. This new approach is simple, nonparametric, and exactly measures the extent of agreement. In addition, flexible clinical tolerance limits can be set in this method – a facility not available with Bland-Altman method.

Assess predictive performance of a model by PPV, NPV, and agreement, instead of the area under the ROC curve

The predictive performance of a model for qualitative outcome is generally wrongly assessed by the area under the ROC curve, called C-index. For example, see Neuman et al¹⁶. This area is based on sensitivity-specificity that assess discrimination of already known outcomes, and not predictive performance for the unknown outcomes. Positive predictive value (PPV) and negative predictive value (NPV) are the right indicators to assess predictive performance but are rarely used. For adequate prediction, both these predictive values must be at least 90 per cent with a 95 per cent lower bound of at least 85 per cent to minimise the error. The cut-off for best predictivity should be where PPV + NPV is maximum, called P-index¹⁷, and not the sensitivity-specificity based Youden index. Secondly, even a low C-index, such as 0.735, has been considered sufficient. Many authors considered C-index value of 0.7 and above acceptable¹⁸. The remaining 0.3 is too much of an error and unacceptable for clinical applications. A stricter value, such as at least 0.9 with 95 per cent lower bound 0.85, should be the norm even for adequate discrimination¹⁹. This may require extra efforts but that is worth it to move from mediocrity to excellence.

For quantitative outcomes, the predictive performance of a model should be assessed again by one-to-one agreement within clinical tolerance between the predicted values for each subject and the corresponding observed values, since clinical applications are not based on averages but on the individual values. A review of the recent 20 articles indexed in PubMed from a variety of journals around the world found that none used this method¹⁹. Flexible clinical tolerance limits can be set in this method, such as correct prediction within 10 per cent of the value in place of an absolute value. This would imply prediction within ±12 min for say, the duration of surgery when the expectation is 2 h and ±24 min when it is 4 h.

Shift from medical statistics to statistical medicine

Statistical methods such as confidence intervals and various tests of hypothesis are commonly used in most empirical research. All these methods come under the generic of medical statistics or biostatistics. An unrealized difficulty with these methods is that they are based on group averages and trends, which can be deceptive when applied to individuals. For example, it can be shown that the regression y = x with 0 intercept and 1 regression coefficient can be obtained even when the values are widely different, since this actually is E(y) = x, where E(y) is the average of y values for a fixed value of x. Many researchers, including biostatisticians, do not realize the potential of misinterpretation of the clinical implications of these group-based statistical methods. Some results fail in actual applications on individuals in clinics because of such inadequacies.

Besides focus on individuals, a new approach is to use statistical tools for diagnosis, treatment, and prognosis in individual patients or persons instead of averages and proportions in groups. These tools are models, scoring systems, scales, indexes, decision trees, and now machine-learning (AI) based approaches. These are being increasingly used for establishing diagnosis, prescribing treatment, and assessing prognosis in clinics at the individual level and have been found to perform better than clinical assessment²⁰. Decision support systems such as improved primary health care²¹ and clinical research calculators, such as for cardiovascular risk²² have shown promise in healthcare of individuals. More such tools and tools with higher validity can be developed to enhance objectivity in clinical assessment. This paradigm shift is called personalized statistical medicine and projected as a branch of medicine, whereas medical statistics is a branch of statistics²³.

Small sample size is better in some cases

Despite advantages of large sample studies, small sample may be more relevant in some situations, particularly for research on biological mechanism of an outcome²⁴. Research by many Nobel prize winners in physiology are glaring examples. For rare disease and health conditions in athletes in elite sports²⁵, small samples are assets. Small sample research in any setup allows use of highly sophisticated instruments and highly qualified professionals to obtain and understand the data. Every subject in the sample can be thoroughly investigated for antecedents, mediators, confounders, and outcomes to establish cause-effect relationship or to explain the biological mechanism. Such a facility is generally not available with large sample studies. A large sample is helpful when multifactorial entities are investigated, where some factors cannot be studied or are unknown, but they are generally for group averages and proportions and not for individual values. The results assume that the effect of unaccounted factors average out. For example, Bogl et al²⁶ conducted a meta-analysis comprising 68,494 same-sex and 53,808 opposite sex dizygotic twins for studying the effect of sex on height in adulthood and found no evidence of a major effect. Perhaps an in-depth study of one pair of M-M, M-F, and F-F with minimal difference in height, and one pair of each with a substantial difference, could provide better evidence. Small sample studies can bring out the details of mechanism – thus helping to tailor the regimen to an individual’s need.

Overall, simple new statistical approaches are now available that enhance the credibility of results, particularly for clinical research studies (Table). These approaches include testing for medically significant effect instead of the null of no effect, using P<0.01 in place of the current P<0.05 for statistical significance for greater assurance, assessing predictive performance of a model by PPV and NPV instead of the area under the ROC curve, using individual agreement within clinical tolerance in place of Bland-Altman method, using statistical tools such as scoring systems for diagnosis, treatment, and prognosis for personalized assessment in place of group averages, and recognizing the importance of small samples in some situations for obtaining better quality data with intensive investigations.

Table. Existing and the newer approaches for better credibility of the results.

Problem	Existing approach	Newer approach
Significance of results	Statistical significance for any effect (H₀: Effect = 0; H₁: Effect ≠ 0)	Medical significance in addition to statistical significance for a minimum medically important effect (H₀: Effect ≤ δ; H₁: Effect > δ)
P value for statistical significance	P<0.05	P<0.01
Paired comparison	Average difference	Difference in individual pairs
Agreement	Bland-Altman method	Percent agreement within clinical tolerance
Predictive performance of a model	Area under the ROC curve (C-index)	PPV and NPV ≥0.90
Adequate discrimination	C-index ≥0.70	C-index ≥0.90
Best cutoff for prediction	Youden index	P-index
Statistical tools	Estimation & testing of hypothesis (Medical statistics)	Models, scoring systems, decision trees for diagnosis, treatment & prognosis (Statistical medicine)
Sample size	Prefer large sample	Prefer small sample for intensive research in some cases

ROC, receiver operating characteristic

Financial support & sponsorship

None.

Conflicts of Interest

None.

Use of Artificial Intelligence (AI)-Assisted Technology for manuscript preparation

The authors confirm that there was no use of AI-assisted technology for assisting in the writing of the manuscript and no images were manipulated using AI.

Reference

Ioannidis JP. Why most published research findings are false? PLoS Med. 2005;2:e124.
[CrossRef] [PubMed] [PubMed Central] [Google Scholar]
Indrayan A. Statistical fallacies errors can also jeopardize life health of many. Indian J Med Res. 2018;148:677-79.
[CrossRef] [PubMed] [PubMed Central] [Google Scholar]
Cohen HW. P values: Use and misuse in medical literature. Am J Hypertens. 2011;24:18-23.
[CrossRef] [PubMed] [Google Scholar]
Wasserstein RL, Schirm AL, Lazar NA. Moving to a world beyond “p < 0.05”. The American Statistician. 2019;73:1-19.
[Google Scholar]
Smith JW. Medical significance of ocular torticollis. Bull Hosp Joint Dis. 1945;6:99-109.
[PubMed] [Google Scholar]
Lin M, Lucas HC, Shmueli G. Research commentary—Too big to fail: Large samples and the p-value problem. Information Systems Research. 2013;24:906-17.
[Google Scholar]
Frank O, Tam CM, Rhee J. Is it time to stop using statistical significance? Aust Prescr. 2021;44:16-8.
[CrossRef] [PubMed] [PubMed Central] [Google Scholar]
Ciapponi A, Belizán JM, Piaggio G, Yaya S. There is life beyond the statistical significance. Reprod Health. 2021;18:80.
[CrossRef] [PubMed] [PubMed Central] [Google Scholar]
Kang H. Sample size determination and power analysis using the g*Power software. J Educ Eval Health Prof. 2021;18:17.
[CrossRef] [PubMed] [PubMed Central] [Google Scholar]
Egbuchulem KI. The basics of sample size estimation: an editor’s view. Ann Ib Postgrad Med. 2023;21:5-10.
[Google Scholar]
Leyrat C, Eldridge S, Taljaard M, Hemming K. Practical considerations for sample size calculation for cluster randomized trials. J Epidemiol Popul Health. 2024;72:202198.
[CrossRef] [PubMed] [Google Scholar]
Chow S-G, Shao J, Wang H, Lokhnygina Y. Sample size calculations in clinical research (3rd edition). New York: CRC Press; 2017.
Benjamin DJ, Berger JO. Three recommendations for improving the use of\n p\n -values. The American Statistician. 2019;73:186-91.
[Google Scholar]
Kassie TD, Yimenu BW, Baye Temesgen G, Shimelash RA, Abneh AA. Differences in the count of blood cells pre-and post-chemotherapy in patients with cancer: A retrospective study (2022) Front Med (Lausanne). 2025;12:1485676.
[CrossRef] [PubMed] [PubMed Central] [Google Scholar]
Indrayan A. Direct use of clinical tolerance limits for assessing the degree of agreement between two methods of measuring blood pressure. South Med J. 2023;116:435-9.
[CrossRef] [PubMed] [Google Scholar]
Neuman I, Shvartser L, Teppler S, Friedman Y, Levine JJ, Kagan I, et al. A machine-learning model for prediction of Acinetobacter baumannii hospital acquired infection. PLoS One. 2024;19:e0311576.
[CrossRef] [PubMed] [PubMed Central] [Google Scholar]
Indrayan A, Malhotra RK, Pawar M. Use of ROC curve analysis for prediction gives fallacious results: Use predictivity-based indices. J Postgrad Med. 2024;70:91-6.
[CrossRef] [PubMed] [PubMed Central] [Google Scholar]
de Hond AAH, Steyerberg EW, van Calster B. Interpreting area under the receiver operating characteristic curve. Lancet Digit Health. 2022;4:e853-5.
[CrossRef] [PubMed] [Google Scholar]
Indrayan A, Mishra S. Assessing the adequacy of a prediction model. Indian J Community Med. 2025;50:739-44.
[CrossRef] [PubMed] [PubMed Central] [Google Scholar]
Grove WM, Zald DH, Lebow BS, Snitz BE, Nelson C. Clinical versus mechanical prediction: A meta-analysis. Psychol Assess. 2000;12:19-30.
[CrossRef] [PubMed] [Google Scholar]
Agarwal S, Chin WY, Vasudevan L, Henschke N, Tamrat T, Foss HS, et al. Digital tracking, provider decision support systems, and targeted client communication via mobile devices to improve primary health care. Cochrane Database Syst Rev. 2025;4:CD012925.
[CrossRef] [PubMed] [PubMed Central] [Google Scholar]
Galarza-Delgado DA, Gonzalez-Gonzalez V, Guajardo-Jauregui N, La Garza JAC, Arvizu-Rivera RI, Elizondo-Benitez MF, et al. Evaluation of novel cardiovascular risk calculators in patients with rheumatoid arthritis. J Rheum Dis. 2025;32:145-7.
[CrossRef] [PubMed] [PubMed Central] [Google Scholar]
Indrayan A. Personalized statistical medicine. Indian J Med Res. 2023;157:104-8.
[CrossRef] [PubMed] [PubMed Central] [Google Scholar]
Berg RMG, Christensen R, Ried-Larsen M. The corruption of power: On the use and abuse of a pre-trial concept. Exp Physiol. 2024;109:317-9.
[CrossRef] [PubMed] [PubMed Central] [Google Scholar]
Skorski S, Hecksteden A. Coping with the “small sample-small relevant effects” dilemma in elite sport research. Int J Sports Physiol Perform. 2021;16:1559-60.
[CrossRef] [PubMed] [Google Scholar]
Bogl LH, Jelenkovic A, Vuoksimaa E, Ahrenfeldt L, Pietiläinen KH, Stazi MA, et al. Does the sex of one’s co-twin affect height and BMI in adulthood? A study of dizygotic adult twins from 31 cohorts. Biol Sex Differ. 2017;8:14.
[CrossRef] [PubMed] [PubMed Central] [Google Scholar]