Translate this page into:
Use, knowledge, & attitudes on core outcome sets among trialists & systematic reviewers in India: A survey
For correspondence: Dr Soumyadeep Bhaumik, Meta-research and Evidence Synthesis Unit, George Institute for Global Health, 308, Elegance Tower, New Delhi 110 025, India e-mail: sbhaumik@georgeinstitute.org
-
Received: ,
Accepted: ,
Abstract
Background & objectives
Core Outcome Sets (COS) are consensus-derived standardized outcomes that improve the transparency, consistency, homogeneity, and usefulness of outcomes. While COS are being increasingly developed, not much is known about their awareness, use, knowledge, and attitudes among trialists and systematic reviewers. This study aimed to examine the use, knowledge, and attitudes about COS among clinical trialists and systematic reviewers in India.
Methods
We conducted an anonymous online survey between August 2023 to November 2023, using a questionnaire hosted on REDCap including trialists and/or systematic reviewers as participants. The survey tool evaluated awareness, use and attitudes on COS, including through a couple of open-ended questions.
Results
A total of 523 trialists/ systematic reviewers participated in this survey. Of these, only 51.4 per cent indicated that they were aware of COS. Only 13.9 per cent trialists and 10.5 per cent systematic reviewers were using them, while mere 6 per cent had been involved in COS development. Lack of available COS for conditions of interest in an Indian context was identified as one of the barriers in its uptake. Even among participants who were aware, many were not clear about COS concepts, and although they expressed positive attitudes about the benefits of COS usage, they also indicated that they perceived COS as restrictive, costly, blocking innovation, and increasing patient burden. We identified health conditions relevant to India, where there is no COS available or where COS is present but were mentioned to be not relevant or valid for the Indian context.
Interpretation & conclusions
As per our knowledge, this study is the largest survey on this domain globally. Through this study several barriers for enhancing COS awareness and uptake were identified indicating the need for resources and systematic efforts to address these barriers in India and globally. We also identify COS which need to be developed or updated to be relevant to Indian context.
Keywords
Attitudes
clinical trials
core outcome set
India
knowledge
meta-research
survey
The science around evidence-informed medicine has developed exponentially over the last three decades, with trials and systematic reviews becoming an integral part of the knowledge ecosystem machinery in the domain. For trials and systematic reviews to be policy and practice relevant, and for meaningful comparisons of benefits and harm to be made, it is essential to choose the right outcomes which should be measured consistently across studies. There is a lack of consistency in which outcomes are measured in trials, arising due to differences in chosen outcomes, specific measures, and methods of measurement1. Heterogeneity and inconsistency of outcomes makes it difficult to compare and pool data for evidence synthesis, thereby reducing the scope for meta-analyses2. Furthermore, without clear and pre-defined outcomes, authors may selectively report outcomes with significant results (reporting bias)3. Core Outcome Sets (COS) are a consensus-derived, standardized sets of outcomes that should be minimally measured in trials on a specific health condition or disease4-6 thus enabling the evidence base to be built faster and be comparable globally. Development of COS usually follow a standardized process involving multiple interest-holders, including patients, to maximize usefulness of outcomes1.
Despite the increasing availability of COS, its uptake has been sub-optimal and varies widely across areas of health7. For instance, in a review of late-phase trials published in 2019-2020; 98 per cent of the included studies did not use COS8. The Core Outcome Measures in Effectiveness Trials (COMET) initiative, a collaborative of people involved in COS development has conducted two surveys on the use of COS, both with a small number of participants (62 and 81 participants, respectively) despite multi-national scope9,10. In this study, we aimed to fill this gap by examining the use, knowledge, and attitudes about COS by clinical trialists and systematic reviewers in India, with the view of enhanced generalisability through a greater number of participants. We took a country-level approach because different factors would influence issues in different countries, and initiatives for capacity building or uptake are better targeted at country-level.
Materials & Methods
An online cross-sectional study was conducted by the Meta-research and Evidence Synthesis Unit of The George Institute of Global Health, India between August to November 2023. The study protocol was approved by the Institutional Ethics Committee of The George Institute for Global Health, and there was no protocol deviation.
Setting and participants
Adult participants (≥18 yr of age) working in healthcare and related fields, involved in conducting trials and/or systematic reviews, and residing in India or who are Indian citizens were included. There were no other restrictions, and the study was conducted online in English.
Recruitment of participants
Participants were recruited by emailing the listed ‘Principal Investigator (PI) or Trial Coordinator’ of clinical trials from the ‘Clinical Trial Registry- India’ database, registered after Jan 1, 2019 (2019 was chosen as cut-off considering e-mail IDs before that might be either irrelevant or inactive), or through social media posts with links to online survey(example through X (Twitter) and LinkedIn) or by snowballing (participants were requested to forward the emails or links to others, eligible in their network). For those invited by email, one reminder email was sent, at least two weeks after the initial mail.
Sample size
A minimum of 100 participants were planned initially for inclusion, to get a better understanding of the knowledge and perceptions towards COS, than prior studies with 62 and 81 participants, respectively, despite them being multi-country in nature9,10. We planned to maximize the number of participants (with no upper limit on recruitment numbers) in order to increase generalisability.
Data collection
Data was collected in an anonymous manner, with no personal information collected from the participants, through a questionnaire instituted through REDCap (online application for managing database and surveys ( https://redcap.georgeinstitute.org.in/ ). The questionnaire included brief demographic questions, followed by questions on awareness, use and attitudes on COS through a mix of continuous, categorical, Likert-scaled (5-point) items and open-ended questions. The questionnaire was pre-tested on a group of seven individuals internally within the institution. Data from tool testing was not included in our final analysis.
The actual questionnaire was preceded by the informed consent page. The consent page described participant rights and researcher contact information was displayed. The survey was terminated for those who did not consent to participate. The survey was anonymous, and no identifiable information was collected.
The questionnaire was designed for maximal efficiency of time for respondents such that participants after filling brief demographic information were asked if they were aware of COS before this survey (a yes/no question). Only those participants who were aware of COS before the survey were asked other questions, on use and attitudes about COS. For others, the survey ended at that point.
Data analysis
Survey responses were collated and analyzed in Microsoft Excel, which included standard descriptive statistics. Descriptive statistics (percentages for COS awareness and familiarity, and median for attitudes towards COS) were used. Open-ended questions were descriptively reported, with content analysis of the free text responses in the open fields, noting frequencies (as noted in other parameters).
Results
Characteristics of the survey participants
There were 533 responses out of which 10 (1.9%) did not give consent to participate in the study (thus terminating the survey at the consent page). The mean age of the 523 participants was 42.91 yr (19-90 yr; median 42 yr). The survey indicated a good gender balance [male (259, 49.52%), female (261, 49.90%), others (0, 0%); prefer not to say (3, 0.57%)].
Most participants were from Karnataka (89, 17.02%) followed by Maharashtra (70, 13.38%) and Delhi (44,8.40%). There were no participants from Dadra and Nagar Haveli, Daman & Diu, Ladakh, Mizoram, Nagaland, and Sikkim. There were 10 (1.91%) participants currently residing outside India (USA-2, Germany-1, Oman-2, Canada-1, UAE-2, UK-1, did not respond-1). Participants reported diverse expertise in conducting trials and systematic reviews within different specialities. Detailed characteristics are presented in supplementary material.
Of the total participants, 358 (68.5%) were PIs (Principal Investigators) or co-PI or site lead of a trial. There were 212 (40.8%) lead/senior authors of systematic reviews of interventions.
Awareness about COS
A little more than half of the participants (269, 51.4%) were aware of COS, while the rest were hearing about it for the first time from this survey. Around 55.02 per cent of PI / co-PI/ site lead of a trial, were aware of COS, compared to 43.63 per cent of people who were not PI/Co-PI/site leads of a trial. Furthermore, 59.43 per cent of lead/senior authors of systematic reviews of interventions were aware of COS, compared to 46.25 per cent of people who were not lead / senior authors of systematic reviews.
Familiarity and use of COS
Familiarity and use of COS among those aware (n=267), is presented in Box 1.
Participants were asked about the conditions for which they wanted to use COS but could not identify one. The list of these conditions and their actual availability which were unavailable to the participants (identified later from COMET database) is shown in table I. Overall, we found that COS were not available for many conditions of relevance to trialists and systematic reviewers of India. Whereas, in few cases, there was lack of awareness about their availability.
| Condition for which participant to use COS but could not identify one | Status of COS availability in COMET database (last searched and links accessed on August 21, 2024) | |
|---|---|---|
| Communicable disease | ||
| 1 | COVID-19 findings in the lung – typical and atypical ones during the initial first wave. | Not available specific to COVID-19 lung manifestations |
| 2 | Tuberculosis | Not available, but being developed ( http://www.comet-initiative.org/studies/searchresults?guid=c72bde98-56b9-4195-95ba-a07ae954861e ) |
| Injuries | ||
| 3 | Amputation of limbs | Not available, but being developed ( https://www.comet-initiative.org/studies/searchresults?guid=6bf0cb02-07bb-4444-baac-dd0e4ff0d2a9 ) |
| Non-communicable disease | ||
| 4 | Acute intracerebral haemorrhage | Not available |
| 5 | Childhood epilepsy | Not available for all types of interventions. COS is available only for specific intervention types ( https://www.comet-initiative.org/studies/searchresults?guid=995dfd44-32aa-4e0e-8a1d-9997a82f641d ) |
| 6 | Chronic kidney disease (multiple responses) | COS available ( https://www.comet-initiative.org/studies/searchresults?guid=d3beb5d4-5d03-4c02-884c-1e29bc0343f1 ) |
| 7 | Chronic low back pain | COS available ( https://www.comet-initiative.org/studies/searchresults?guid=08e19bb5-a554-49b4-b28e-c99666ae882a ) |
| 8 | COPD | COS available ( https://www.comet-initiative.org/studies/searchresults?guid=daefbd7e-a062-4ca6-8f2b-7bb1be766e81 ) |
| 9 | Degenerative conditions of musculoskeletal system | Broad domain mentioned- not searched. |
| 10 | Diabetic retinopathy | Not available, but being developed ( https://www.comet-initiative.org/studies/searchresults?guid=60cbfea6-916c-4b01-9630-e1875bb88dd5 ) |
| 11 | Endocrine disorder | Broad domain mentioned- not searched. |
| 12 | Iron Deficiency anaemia | Not available, but being developed ( https://www.comet-initiative.org/studies/searchresults?guid=2330d43a-d6dc-45fb-bf36-8642e79df1e8 ) |
| 13 | Knee osteoarthritis | COS available ( https://www.comet-initiative.org/studies/searchresults?guid=6eed012e-f6ad-4840-b75d-81217fb6f528 ) |
| 14 | Multiple sclerosis | COS available ( http://www.comet-initiative.org/studies/searchresults?guid=cdc5a4fb-f692-416d-af88-7261130bd2c4 ) |
| 15 | Palliative medicine | Broad domain mentioned- not searched. Several COS in the domain is available |
| 16 | Pancreatitis | Not available, but being developed ( https://www.comet-initiative.org/studies/searchresults?guid=fc8048f2-8777-444d-ba96-6704ed231927 ) |
| 17 | Polycystic ovarian syndrome | COS available ( https://www.comet-initiative.org/studies/searchresults?guid=1ccbbfb5-699b-40b2-b0d7-5475f328670e ) |
| 18 | Pregnancy | Available for many complications of pregnancy but not for normal pregnancy ( http://www.comet-initiative.org/studies/searchresults?guid=1b75f4c4-7ee5-4d52-9dd3-2f04ae6a2590 ) |
| 19 | Reproductive health | Broad domain mentioned- not searched. Several COS in the domain is available |
| 20 | Rheumatoid arthritis | COS available ( http://www.comet-initiative.org/studies/searchresults?guid=d88d65d7-76b9-4885-bd72-d0502b88e796 ) |
| 21 | Sickle cell anaemia | COS available ( http://www.comet-initiative.org/studies/searchresults?guid=6adca623-1651-4697-90b5-02b1b9eea4d5 ) |
| Oral health | ||
| 22 | Dental caries (multiple response) | Not available, but being developed ( https://www.comet-initiative.org/studies/searchresults?guid=1de72fc1-9f96-4806-bdbe-a24b5c113429 ) |
| 23 | Gingivitis | Not available ( https://www.comet-initiative.org/studies/searchresults?guid=57896287-1a99-423b-bb9a-73f1e39de748 ) |
| 24 | Oral health | Broad domain mentioned- not searched. Some COS in the domain is available |
Note: Multiple participants noted lack of availability of COS on Ayurveda interventions. The system for Ayurveda is different from modern medicine. There is an epistemological gap around understanding whether condition-specific COS, which are already available can be used for Ayurveda interventions too. COS, core outcome sets
Participants mentioned that for the following diseases/health conditions available COS was not relevant or valid in the Indian context/setting:
Communicable disease like Tubercular meningitis;
Non communicable diseases including kidney diseases (multiple responses including one mentioning CKD specifically), childhood epilepsy; youth mental health interventions to improve positive mental health among young people in the school/college/community settings; comparing two drugs for analgesic efficacy; other conditions reported were rhinosinusitis; and systemic sclerosis.
Knowledge about COS concepts were assessed among individuals who were aware of COS. We asked several statements with respect to COS concepts, but deliberately refrained asking it in a true-false format (in which guesswork could lead to a 50% correct response). Instead, we used a 5-point Likert scale of agreement to understand if the participants to choose the correct response (strongly agree to the statement). We found that majority of the participants had some misgivings about the core concepts of COS (Table II).
| Statement in survey to assess knowledge of COS | Participants who said strongly agreed to statement (n)/Total number of respondents (N); % |
|---|---|
| All outcomes in the COS should be measured in a trial | 71/214; 33.2 |
| All outcomes in the COS should be used in a systematic review | 61/213, 28.6 |
| Other outcomes can be measured in addition to outcomes in COS in a trial/systematic review | 58/211, 27.5 |
| COS reduce heterogeneity in how outcomes are measured | 48/211, 22.7 |
| COS reduces outcome reporting bias | 72/213, 33.8 |
| Development of a COS involves multiple stages | 86/213, 40.4 |
| Using COS increases comparability of findings across trials | 78/213, 36.6 |
Attitudes about COS among participants who were aware were also recorded. Majority of participants who were aware of COS, had positive attitudes towards COS, in terms of improving transparency, openness, interest-holder relevance and being relevant to routine care. However, substantial proportion of participants believed that COS were restrictive in nature, added to the cost of trial, led to patient burden, and prevented trialist innovation (Table III).
| Statement in survey to assess attitudes about COS | Strongly disagree; n,% | Disagree; n,% | Neither agree nor disagree; n,% | Agree; n,% | Strongly agree; n,% |
|---|---|---|---|---|---|
| COS increase transparency and openness of research | 5, 2.3 | 3, 1.4 | 12, 5.6 | 99, 46.5 | 94, 44.1 |
| COS is restrictive in nature | 8, 3.8 | 64, 30 | 79, 37.1 | 52, 24.4 | 10, 4.7 |
| Using COS adds to cost of trial | 13, 6.1 | 46, 21.6 | 68, 31.9 | 70, 32.9 | 16, 7.5 |
| Using COS adds to patient burden (quantity and repetition) | 14, 6.6 | 69, 32.4 | 60, 28.2 | 57, 26.8 | 13, 6.1 |
| Use of COS increases relevance of outcome to key stakeholders (including patients) | 2, 0.9 | 12, 5.6 | 13, 6.1 | 101, 47.4 | 85, 39.9 |
| COS prevent innovation for trialists | 24, 11.3 | 64, 30.2 | 65, 30.7 | 49, 23.1 | 10, 4.7 |
| COS is relevant to routine care | 4, 1.9 | 12, 5.6 | 34, 16 | 114, 53.5 | 49, 23 |
Discussion
This study demonstrated that almost half of Indian trialists and systematic reviewers were not aware of COS at all. Even among those aware, uptake of COS in trials and systematic reviews was poor (13.9% and 10.5%, respectively). This study also identified four barriers to COS uptake, namely, a) unavailability of COS for a number of conditions of relevance to trialists and systematic reviewers of India or available COS perceived to have low relevance; b) low knowledge about core concepts around COS; c) negative attitudes around COS being restrictive, preventing innovation, adding to patient burden and trial costs; d) for many health conditions relevant to India, there was no COS. Furthermore, it was observed that participants were not aware of the COS-despite its availability, or available COS was thought to be irrelevant or invalid in the context.
The first study on this topic, which had just 62 respondents (with >75% participants from Europe), identified poor knowledge about COS and negative perceptions (particularly about COS being restrictive and often containing too many outcomes) as a major barrier to COS uptake10. While we found around 50 per cent participants were aware of COS in this study, and the uptake of COS in trials was only 13.9 per cent, the corresponding figures in the aforementioned study were higher at 65 per cent and 50 per cent, respectively. However, the small sample size of the previous study limits interpretative comparison. Both the present study as well as Bellucci et al10 study identified similar negative perceptions, but our study identified additional issues around non-availability of COS for health conditions of relevance to India and the low relevance of available COS. These aspects are related to interest-holders from low- or middle-income countries (LMICs) (including India) not leading to the development of COS and being excluded in COS development despite its supposedly global scope. The poor involvement of interest-holders from LMICs in COS development has been documented earlier11-14.
The more recent COMET study was among 81 LMIC interest-holders9. The study used a case-study approach preventing direct comparison. However, the quantitative part of the study reported lesser awareness (32.09%), but similar COS uptake (12.34%). The study found that usefulness, availability, and practicability of COS in LMIC settings and poor involvement of patients and carers in their development were key barriers to COS uptake. The present study using a quantitative approach, found similar challenges around usefulness, availability, and practicability of COS in the Indian setting, as well as additional aspects. Although these barriers do highlight drawbacks of COS, some perceived barriers are also reflective of decreased knowledge regarding COS. For instance, although many participants indicated the COS were restrictive, only 27.5 per cent of participants understood that other outcomes can be measured in addition to outcomes in COS in a trial/systematic review.
As per our knowledge, the present study is the largest survey so far which provides insights about barriers for COS uptake, despite its scope being limited to a single country (India), unlike previous studies which had multi-national scope9,10. This is a key strength of the survey. Considering the fact that English is the lingua franca of academic medicine and healthcare in India, the choice of language had no major implications on representativeness of the survey results in the context of this study. Many e-mails sent to trialists’ e-mail addresses from CTRI (Clinical Trial Registry of India), bounced back or were undelivered. This method of electronic recruitment may have resulted in researchers who were already familiar with COS being more likely to respond to the survey. We thus envisage that our results paint a more optimistic picture than the reality. This is a perceived limitation. Implications of the study for policy, practice and future research is summarized in Box 2.
In conclusion, there is a need to enhance the awareness of the concepts and importance of COS to facilitate higher uptake in research. Initiatives can be taken to increase understanding of how COS can improve data comparison to drive evidence-based policy and practice. COS development should also promote inclusion of interest-holders across countries if COS are to be relevant in a global context.
Acknowledgment
None.
Financial support & sponsorship
None.
Conflicts of Interest
SB is as a member of the COMET POPPIE Working Group ( https://www.comet-initiative.org/Patients/POPPIE , accessed on August 21, 2024). This is a voluntary role. The author(s) declares no other potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Use of Artificial Intelligence (AI)-Assisted Technology for manuscript preparation
The authors confirm that there was no use of AI-assisted technology for assisting in the writing of the manuscript and no images were manipulated using AI.
References
- Developing core outcome sets for clinical trials: Issues to consider. Trials. 2012;13:132.
- [CrossRef] [PubMed] [PubMed Central] [Google Scholar]
- Heterogeneity of primary outcome measures used in clinical trials of treatments for intermediate, posterior, and panuveitis. Orphanet J Rare Dis. 2015;10:97.
- [CrossRef] [PubMed] [PubMed Central] [Google Scholar]
- Choosing important health outcomes for comparative effectiveness research: A systematic review. PLoS One. 2014;9:e99111.
- [CrossRef] [PubMed] [PubMed Central] [Google Scholar]
- Core outcome sets and systematic reviews. Syst Rev. 2016;5:11.
- [CrossRef] [PubMed] [PubMed Central] [Google Scholar]
- Outcomes in intervention research on snakebite envenomation: A systematic review. F1000Res. 2022;11:628.
- [CrossRef] [PubMed] [PubMed Central] [Google Scholar]
- The COMET Handbook: Version 1.0. Trials. 2017;18:280.
- [CrossRef] [PubMed] [PubMed Central] [Google Scholar]
- A systematic review finds Core Outcome Set uptake varies widely across different areas of health. J Clin Epidemiol. 2021;129:114-123.
- [CrossRef] [PubMed] [PubMed Central] [Google Scholar]
- Use of core outcome sets was low in clinical trials published in major medical journals. J Clin Epidemiol. 2022;142:19-28.
- [CrossRef] [PubMed] [Google Scholar]
- Awareness and experiences on core outcome set development and use amongst stakeholders from low- and middle- income countries: An online survey. PLOS Glob Public Health. 2023;3:e0002574.
- [CrossRef] [PubMed] [PubMed Central] [Google Scholar]
- A survey of knowledge, perceptions and use of core outcome sets among clinical trialists. Trials. 2021;22:937.
- [CrossRef] [PubMed] [PubMed Central] [Google Scholar]
- Inclusion of participants from low-income and middle-income countries in core outcome sets development: A systematic review. BMJ Open. 2021;11:e049981.
- [CrossRef] [PubMed] [PubMed Central] [Google Scholar]
- Systematic review of international Delphi surveys for core outcome set development: Representation of international patients. BMJ Open. 2020;10:e040223.
- [CrossRef] [PubMed] [PubMed Central] [Google Scholar]
- Development of a core outcome set for multimorbidity trials in low/middle-income countries (COSMOS): Study protocol. BMJ Open. 2022;12:e051810.
- [CrossRef] [PubMed] [PubMed Central] [Google Scholar]
- Addressing the burden of snakebite: analyzing policy prioritization, evaluating health systems, and fostering research on treatments. University of New South Wales; 2023. Doi: https://doi.org/10.26190/unsworks/25043
- [Google Scholar]
