Publication-equivalence: Conceptually sound but practically challenging

Sir,

I read with interest the perspective article titled ‘Publication-Equivalent as the New Single Currency of Research Impact: The ICMR-Impact of Research and Innovation Scale (ICMR-IRIS)’ authored by the Director General of the Indian Council of Medical Research, New Delhi¹. This article is both timely and thought-provoking. It touches upon an issue that has been long debated across the scientific world—how best to measure the true impact of research. While metrics such as impact factor, citation count, or the h-index have been widely used, they remain imperfect and often fail to capture the real translational and societal contributions of scientific work. The proposed ICMR-IRIS and its construct of the publication-equivalent (PE) addresses these long-standing gaps to a great extent and deserve careful attention and deliberation.

For decades, the scientific community has grappled with the challenge of how to assess research output in a manner that is objective, transparent, and fair across disciplines. Funding agencies, research institutions, and even national governments require measurable indices to evaluate the performance of their researchers, to justify investments, and to promote accountability. Traditionally, the ‘impact’ of a researcher has been evaluated through parameters such as the impact factor of journals in which they publish, or by citation-based indices such as the h-index or i10 index. While these numbers have been widely adopted, they are also widely criticised.

Impact factor, as we know, is a journal-level metric. Although it is frequently used as a proxy for article quality, this may be misleading. Thus, while impact factor remains a convenient shorthand, it is by no means a reliable indicator of the value or influence of individual research contributions.

Citation counts and related indices, such as the h-index, attempt to shift the focus from journals to individual articles or researchers. These are equally fraught with limitations². The reliance only on citation counts creates distortions, rewarding some disciplines disproportionately while undervaluing others.

Commercial players such as Clarivate (through Web of Science) and Elsevier (through Scopus) have entrenched these systems further by offering their own proprietary impact indicators^2,3. Both indices pride themselves on upholding quality standards—peer review, publication regularity, and ethical practices—but both remain fundamentally tied to the business interests of their parent organisations. Many medical researchers are trained to publish in journals that are indexed in PubMed to attract maximum readership from the medical fraternity. But MEDLINE does not have any scoring system. Moreover, the National and international ranking agencies, including India’s National Institutional Ranking Framework (NIRF), emphasises more on Scopus-indexed journals, despite the fact that Scopus indexing does not limit a journal to remain within a focused subject area.

Recognising the shortcomings of already available metrics, newer approaches have emerged. For example, Scopus, in collaboration with Elsevier and Stanford University, has introduced the c-score for author ranking, identifying the top two per cent scientists globally since 2020⁴. This system takes into account multiple parameters: total citations, h-index, authorship position, and single-author contributions, among others, yet it is imperfect. Notably, patents, translational research outputs, and innovations with social or policy impact are left unmeasured. This is one reason why, despite India’s large medical research community, relatively few clinicians or biomedical scientists appear in these top two per cent rankings, while fields such as physics or computer science dominate.

This is where the ICMR-IRIS framework, published in the IJMR, assumes particular relevance. As Dr. Bahl has noted, that the ultimate value lies not in the number of citations but in the ability of research to generate patents, to yield technologies, or to influence health systems and policies. Translational value, societal benefit, and innovation outcomes are arguably more meaningful than publication counts, yet they remain under-recognised in existing systems.

The idea of a publication-equivalent metric is thus both innovative and necessary. By assigning weightage to diverse performance indices, ICMR-IRIS moves beyond narrow bibliometrics. However, for this to gain traction and acceptance, it is important that the system be generalised rather than remain ICMR-specific. With appropriate refinements and consensus-building, such a metric could be adopted not only by other funding agencies in India but potentially even at the global level.

That said, several challenges remain. While the PE formula is appealing for senior researchers with established track records, it may prove less suitable for young scholars just beginning their careers. Early-career researchers may lack patents or translational outputs, but this does not mean their contributions are any less valuable. Safeguards are needed to ensure that the system does not inadvertently disadvantage or demotivate the next generation of scientists. Inclusion of case studies, worked examples, and transparent scoring frameworks would be immensely helpful in clarifying how PEs are to be assigned across diverse scenarios.

Without clear algorithms or almetrics, ambiguity will remain. Similarly, negative factors such as retracted papers, data manipulation, or conflicts of interest must also be accounted for, lest the metric reward quantity without safeguarding integrity.

Another important question is the time frame for assessment. Should PEs be calculated annually, over rolling 2–5-year windows, or over an entire career? Each option has its merits and drawbacks. Annual assessments may favour shorter-term outputs, while lifetime scoring risks over-rewarding those who were productive decades ago but are no longer active. A tiered system may be worth exploring.

In conclusion, the ICMR-IRIS represents a forward-looking attempt to redefine research impact in terms of publication equivalents that integrate publications, citations, patents, innovations, and societal contributions. While the framework is promising, it will benefit from broader consultations with the scientific community, transparent algorithms, and pilot testing with real-world case studies. Importantly, it must balance recognition of senior scientists with fairness to early-career researchers. If successfully developed and implemented, such a system could serve as a model not just for India but globally, helping to move the scientific world beyond the narrow and often misleading metrics of impact factor and citation counts, toward a richer, more meaningful measure of research excellence and innovation.