Beyond the Publication-Equivalent: Rethinking research impact metrics

Sir,

An important article by Bahl R. on the ICMR - Impact of Research and Innovation Scale (ICMR-IRIS) published in the August 2025 issue of the Indian J Med Res introduces a much-needed framework for evaluating research outcomes in India through the concept of the Publication-Equivalent (PE)¹. This initiative is timely and commendable, as India continues to expand its biomedical research ecosystem and requires robust tools to measure not just productivity but also effectiveness of scientific investments. Beyond accountability for public funding, such metrics can help shape evidence-driven policy decisions, improve individuals as also institutional benchmarking, and guide capacity-building.

To strengthen its practical utility and help ICMR-IRIS better capture the layered nature of research influence over time, it could integrate weighted impact tiers rather than fixed PE values for all indicators. For example, a guideline change that influences national policy could have a higher tier (e.g., 15 PE) compared to one affecting a local programme (e.g., 8 PE). Such tiered weighting would reflect the depth and breadth of impact. Research impact evolves gradually, often through incremental steps, as seen in models like the Payback Framework and Research Impact Framework^2,3.

The PE scale would benefit from validation across multiple datasets, such as comparing PE scores with citation impact, patent utilization, and health programme adoption. A multi-agency validation, including Government as well as privately funded projects, would ensure representativeness across disciplines and funding bodies. Approaches similar to the Translational Research Impact Scale (TRIS) provide useful precedents for establishing construct validity and reliability⁴.

The scale could incorporate a few qualitative indicators that capture non-quantifiable dimensions of impact, for example, documented use of research in government policy briefs, contributions to training or workforce capacity, and evidence of stakeholder engagement in project design. These indicators would complement quantitative outputs and reduce bias against disciplines like epidemiology, social medicine, and implementation science, where outcomes are often long-term and context-dependent^5-7. Similar to the UK Research Excellence Framework and the Australian Research Council’s Engagement and Impact Assessment, ICMR-IRIS could adopt a hybrid assessment model combining numeric indicators with narrative case studies^8,9.

To promote fairness in collaboration, contribution-based weighting could be introduced, such as assigning primary investigators a larger PE share (e.g., 40–50%) while proportionally distributing the remainder among collaborators. Transparent contribution matrices, as used in authorship models (CRediT taxonomy), could serve as templates¹⁰. Additionally, using PE in addition to other evaluation tools can help prevent metric-driven behaviour and uphold research integrity.

Periodic recalibration of PE values could allow this metric to adapt to emerging research priorities and evolving definitions of impact. Flexibility would be particularly beneficial for fields like digital health or genomics, where the pathway from discovery to societal benefit is rapidly changing.

The ICMR-IRIS framework represents a promising move towards more systematic and transparent research evaluation in India. Its simplicity makes it useful for comparison and accountability; however, overreliance on numerical indicators may overlook the broader scientific, social, and translational value of research. By integrating empirical validation and qualitative assessments, this framework could evolve into a comprehensive model that not only measures output but also captures real-world relevance.