In conversation with Dr. Rajiv Bahl: The man behind the radical transformation of ICMR

SPOTLIGHT: Dr. Rajiv Bahl

Dr. Bahl, since your association with Indian Council of Medical Research (ICMR), metric-mediated expert selections, project evaluations, etc., have been charting the talks. What are your thoughts and experiences behind mobilizing such radical transformations?

The changes that we have instituted so far are several-fold. One crucial change, as you mentioned is related to ‘Who are our experts?’ Whether these are a members of our Scientific Advisory Committees (SACs) or Project Selection Committees or Project Review Committees (PRCs)! So, what we have done in the past three years is to make the process more transparent as well as evidence-based. For instance, previously research experience was not objectively assessed in the true sense for any Scientist. It was felt that people who are assessing their work are not themselves truly researchers despite being Heads of Department of Medical Colleges or even National Institutions. What we did for that is we asked our people to create a one page CV, where we looked into, of course, the expertise area, and the experience but more importantly we looked at the H-index, meaning the citations of the chosen expert’s research. This was to ensure that the chosen expert has some reasonable level of importance. We presumed that none of the chosen experts, under normal circumstances, have an H-index of <20. Only a couple of exceptions, where people who do a lot of patents and are innovators, may not have the H index as high, so their patents are looked at. Furthermore, in certain instances, we may include experts who are more programme oriented. This helped us change our SACs to make them more national. Previously the SACs comprised of experts only from the respective city or in and around the respective State. But now, wherever the Institute may be, the SAC is selected from anywhere and that has brought national character to our Advisory Committees.

For our PRCs, we decided that we will minimise conflicts of interest. Therefore any proposal that is received from a given institution is not evaluated by an expert from that institution. The second change was to create a mandatory declaration form where financial, academic, personal conflicts of interest (people who may be relatives, close friends, etc., or have worked together closely) are to be declared to be able to participate in such committees. Previously, a large number of projects were being selected from institutions from where we were drawing most of our experts. Now we have a wider spread of selected institutions all across the country. Of course there are good institutions with a long history such as PGI Chandigarh, AIIMS, New Delhi, etc that still take a lot of projects, but now we ensure that this is based on merit alone.

The third change that you talked about is a mix of all that I’ve discussed. It is about going beyond publications and even beyond citations, because we do not do research only for citations. We try to do research because we are not ‘only’ doing basic science, but we are also doing Development, some Basic Science, Descriptive research, and some Delivery research. For this we have created a scale where we convert all the impact into a ‘Publication Equivalent’ (PE). But the idea is to give more weight to those things which actually have a public health impact and/or a clinical impact. For instance, if a research finding is published, it gets 1 PE, However, if it has some public health impact or changes public health practice or programme/clinical practice then it gets 10 PE, so weight is 10-fold. A paper which gets very high citations gets 2 PE in addition to the 1 that is the publication itself. The idea is to give more weightage. Similarly, innovations, new technologies, commercialised technologies get more weightage. Five PE for patents, 10 for technology development are awarded and 20 if it reaches the masses, through its commercialisation.

The last thing is to say is that the idea behind this scoring system to instil among people who are writing or evaluating proposals and those who are conducting the studies, that eventually they must produce outcomes. The goal cannot just be a publication! This is the reason to go in that direction which I am pretty confident that will make the overall Indian health and medical research more impactful.

Thank you, Sir! You’ve already detailed about the ICMR-Impact of Research and Innovation Scale (IRIS) and the Publication Equivalents. This has been generating quite a stir among researchers since its recent publication in the August issue of the IJMR. Pulling onto the same thread, could you share about what the felt need was for coming up with this scoring system and do you think there is room to buff it up any further?

The felt need was that we reviewed all the ways to measure ‘impact’. Be it impact of a project, impact of an institution, impact of ICMR and what I found was that we only have multi-indicator based impact. For instance, a researcher may have 150 publications and one patent while another may have less number of publications and more patents, some other may only have publications. Now the world’s top 2 per cent scientists list is decided based only on publications and citations. So, if we have to incorporate scientific impact along with societal impact of research, how could we do it with a single indicator? Now whenever we will do this, there is going to be some concerns of undervaluing A or overvaluing B or undervaluing C. However, I would like people to focus on the concept. The concept is that if you publish you get something. But if your publication leads to a large number of citations, you get more, which is a measure of scientific impact. If the same changes either policy, strategy or practice, in clinical care or in public health, you get 10 times more credit than the publication. Similarly, if you create an innovation through research and that innovation is used by a larger number of people, then it is of very high impact.

Another point is that, often this question is asked by funding agencies as well as the political leadership, ‘What has India done?’ ‘What new medicines, new vaccines, new tests has India produced?’ So if we just keep talking about academically A is better, why A is better, it might have a better impact 50 years later. Sure! Then you will get that impact, whenever that happens. If a basic research led to something, it should also be given due credit. However, the idea here is not to punish anybody or reward anybody. The idea is to change the mind-set. The mind-set should be that ‘I must produce something from this which is worthwhile! Worthwhile for science and/or worthwhile for society’. If one is not doing anything which is worthwhile then it is only curiosity driven! And there is a space for that! The Indian Council of Medical Research, is a part of the Ministry of Health and Family Welfare. Our goal is to improve the health of people. We do not have the luxury of just saying ‘we will do something and we will see what happens later!’ We want this investment to make a change, and our investment is limited. We do not have 44 billion USD like National Institutes of Health (NIH). We have 500 million USD which is roughly 1/80^th of what they have. So we must do it in a way that we encourage people to generate a greater impact. That is the ultimate goal!

When you say, Can we improve it? Yes! Everything can be improved, and we will continue to improve it as we use it. It is not laid in stone. If tomorrow we think that some type of basic research publications should have even a little more impact, because there are fundamental concepts that have been changed, by all means we will do that. Similarly, if we think that public health impact that has actually saved millions of lives deserves to have more than 10, then we will add that. Nobody is stopping us. We have 8 indicators, we have weights for each. We can make 9 or 10 indicators. As we use more and more of this scale, we can change weights if required. Instead of 10, something could become 15 or instead of 5 something could become 3. It will depend upon what the collective thinking of people is. I also believe that we do not want to average out, that everything is the same, and one way of averaging out is taking everybody’s opinion. In this case, a basic scientist will ask for a very high value on basic sciences or a developer of innovation may ascribe a very high value to innovation, so, I think this should be done by policy makers alone.

So, this consultation that we have done was that we have talked to people in Policy, for example, Department of Health Research (DHR)-Secretary, Health Secretary, National Institute of Transforming India (NITI) Aayog or Office of Principal Scientific Advisor (PSA) and that is a policy decision that the Government of India today will give more value to this. I’ll give you this example. In the USA, NIH generally focuses only on basic research. They will not fund clinical trials, they will not fund clinical studies, they will not go into funding descriptive research either. Why? Because Centers for Disease Control and Prevention (CDC) does most of the funding for descriptive work and the industry funds most of the development research. All, almost all of the development research from A to Z is funded by industry except for the basic science part. So there it makes sense for the Government to do that. Why? Because the industry is investing in development research. In India we have a very different kind of relationship between industry and research. It is in the growing phase. Industry is still risk averse. It does not want to simply invest in high risk ventures. When a pharmaceutical multi-national company invests in a drug molecule, at the very beginning at the Technology Readiness Level (TRL)-3 stage for example, the chances that it will be licensed by the U.S. Food and Drug Administration (USFDA) would be less than 1 in 10, which means that 9 out of 10 funding efforts will be wasted. However, the system there is such that the one that succeeds can put almost any price on that propriety drug. So, innovation can be sold as very expensive because they have to recover the cost for the 9 things that did not work. Can we do that in India today at this stage? If a medicine costs INR 1 Cr per year, how many people will be able to pay for it? If a vaccine costs INR 10,000 per dose, how many children can get that vaccine? So, we cannot at this time have the same model of innovation. Therefore we must fund research and we must give more credit to research which is beyond basic as well as fund basic research and give its due credit. This is the need of the country, and that is a policy maker’s decision. So, our thinking is based on ‘what does India need today?’ In 20 years this metric may shift by giving more value to basic and fundamental things than it is giving today and that is exactly what I want to explain.

I have done hundreds of Delphi or CHNRI like processes. We could do that, but this does not require that. This is more on how do we fuel the engine of innovation in this country and make India Aatma Nirbhar and make India Viksit Bharat. This is the need of today. From 2047 we might think completely differently. So this scale may also shift.

Do you think metricising researcher performances and outcome evaluations would be perceived as a felt inspiration to perform better or do you foresee some dissuasion among those whose research funding could be curbed owing to their past performances?

Firstly, anybody who is associated with research must look at evidence in its completeness. If you look at all the social media messages or all the opinions that have been raised on ICMR-IRIS, around 80-90 per cent of them are highly lauded and positive. 10-20 per cent people would like some things to be changed and may be 5-10 per cent may not want this at all, because they think this should not be done. Everybody is free to have an opinion, but if you do a poll of people across the country and ask, ‘do you want a more holistic evaluation of this metric?’ I would say an overwhelming majority will agree with us. So, one has to look at it in perspective. The other thing that I have learnt in my 30 years of working is that anything that is important will create a debate and will create people for and against. Our question is that, do we have 80 per cent -yes and 20 per cent -no or 40 per cent -yes and 60 per cent -no, and that we will see. We will look at what time will tell. My view on this is that any policy person I have shown this to has found this to be an exceptional idea. Whether it is NITI Aayog or previous Director Generals of ICMR or others, DBT secretaries, other previous Secretaries, everybody has said this. The whole Performance Evaluation Committee (PEC) of ICMR for example, unanimously felt that this is probably a very good idea. So it is not about everybody liking something. I have no qualms about some people not liking something. We are a democracy, do you ever have 100 per cent votes for one party? Never! One may have opinions, choices and different thoughts. We encourage the thought to be different. That is the whole idea. How can this dissuade anybody from doing research? I don’t even understand that. I read a newspaper article that said it will reduce public health research and I thought, why? If we are giving 10 times more credit to something that actually supports public health or changes practice, why would it dissuade people to do public health research? So, some things are difficult, some things are doable. For example, are we giving more credit to the latter part of the pathway which I have explained, towards development, translation, actual change? Yes, we are, and that’s a policy decision! Why? Because I know we have been doing a lot of basic projects and leaving them in the middle. And I want people to take it to the end. So, to say that it will dissuade anybody from doing research, I find that very difficult.

The only type of research projects we have said we will fund less, and less means about 25 per cent. ICMR funding today is 25 per cent for Discovery research, 25 per cent for Development research, 25 per cent for Delivery research and 25 per cent for Descriptive research. That’s our stated goal. Sometimes we invest a little more in one type of scheme, sometimes in another type of scheme. But our goal is to balance the portfolio. Why do we need to balance the portfolio? Because in 2022 when I looked at the portfolio, 70-75 per cent of all the funded research was Descriptive. Is just understanding the problem enough? Or should we help try solve the problem as well? I believe in the latter! So, I don’t know in which way it will dissuade anybody. The only thing it can change is that what type of research we should do. One that has more value to society? Sure, and if this dissuades people form doing research that has no value for the masses, then I have no problem with that.

Globally given the changing publication landscape with transformative agreements gaining their grasp, there are several Journals which are renouncing ‘the impact factor’. What are your views on this?

Only laying stress on Impact Factor, I am in agreement that I am against it. Is it only Impact Factor that matters? Absolutely not! Secondly there has been a democratisation by journals becoming more accessible online, with open access options available. These days it should not matter a great deal where you publish. What is more important is that it should go into public domain. But to say that Impact Factor has no meaning is also not correct. Because what is the other way you measure? You are also going to do citations eventually and what is Impact Factor? It just says what is the average number of citations a publication got. Should a journal like the Indian Journal of Medical Research (IJMR) not try to have a greater Impact Factor? It means, your papers are readable. More researchers are reading and referring to them. Those are important papers. It’s like saying are marks important in grade 10 or 12 is like saying that someone got 80 or 90 or 100 per cent marks is important? I would say if you only look at 99 is different from 98 and marks are the only thing that you are working towards, it’s not correct. But if you have no grading system at all, not even A+ or A, B, C, D that also does not lead anywhere. So there needs to be some balance.

Lastly Sir, talking about the IJMR, your bold vision to ‘Make IJMR Count Again’ so to say had deployed enthalpy to say the least and is hitting phase change now. How does it look from your current vantage point and where do you wish to see this Journal in the next ten years?

I think we are just starting. The idea for IJMR to be a good journal is that, there are only two things that I have tried to change, that we must be quick, we cannot be slow. An article took more than two years to get published at one point, so whether it is screening, peer-review, publication, actual processing all that needed to change, and it has changed, substantively! It’s just a few months now 3-4 months as compared to two years earlier. I want it to be very short. If you want to reject something it should not take more than 2-3 days. Something is peer-reviewed, then within six weeks we should be able to get back to the authors. So that’s the one side. But the more important side is to see if can we get people to submit good quality papers to IJMR. One influencing factor for this is its low impact factor as compared to some other Journals. So more interesting publications we can get in this it’s either an upwards spiral or a downward spiral. There’s no static in this. So either we will go upwards or downwards and the idea is to go upwards and I think we have started.