Cross-cultural standardization of the South Texas Assessment of Neurocognition in India

In recent years there has been increasing emphasis on the role of cognitive function in mental illness. The impact of mental illness on cognition has long been recognised and cognitive “symptoms,” particularly relating to attention, memory and decision-making are included in operationally defined diagnostic criteria for affective, anxiety and psychotic disorders and dementia12. Currently, cognitive impairment in mental illness is considered mostly in terms of its contribution to pathophysiological processes underpinning disease expression and in relation to treatment and functional outcomes.

Unlike clinical symptoms, measures of cognitive task performance are more objective and reliable. Additionally, changes in cognitive tasks that are putatively sensitive to dysfunction within specific brain circuits can inform neurobiological models of mental disorders. This appears particularly relevant in the context of increasing success in identifying risk genes for most of the major mental illnesses34 as cognitive dysfunction is thought to index processes that mediate between genetic influences and clinical symptoms. Cognitive deficits are also important determinants of functional outcome in patients with mental illness. This has been firmly established for patients with schizophrenia5 and is likely to apply to patients with Bipolar Disorder6.

Despite the central role of cognition for mental illness there is limited research in this area from low and middle income countries. Cross-cultural validation of cognitive testing is, therefore, a key priority but to date this has been achieved only in the field of dementia through the efforts of the 10/66 Dementia Research Group7 and to some extent in schizophrenia6. The primary objective of this study was to examine the applicability of using cognitive tests from the Java Neuropsychological Test (JANET) battery, a computerised version of the South Texas Assessment of Neurocognition (STAN), in India. The JANET includes tests selected on evidence of significant heritability and sensitivity to schizophrenia, bipolar disorder and other brain-related illnesses. The validity and reliability of these tests have already been established and these have been used for large scale studies in North and Latin American populations thus establishing their utility in samples with variable cultural and linguistic background8. Additionally, the availability of these tests in computerized form minimizes rater bias and measurement errors. In this study a subset of the JANET battery was employed selecting those tests that are largely language independent focusing on general intellectual ability, memory, attention, response inhibition and emotional processing.

Material & Methods

This is a collaborative study between the Institute of Psychiatry, King's College London, UK, the Amrita Institute of Medical Sciences (AIMS), India, and Olin Neuropsychiatry Research Center at the Institute of Living, Yale University School of Medicine, USA.

Indian sample: Indian sample was collected over a six month period of January-June, 2010. Healthy volunteers were recruited from the staff employed by AIMS following internal advertisement. Eligibility was based on the following criteria (i) male or female, (ii) native to the state of Kerala, India, (iii) aged between 18 to 65 yr, (iv) minimum of 10 years of education to ensure that participants would be familiar with using computers, (v) not fulfilling current or lifetime Diagnostic and Statistical Manual of Mental Disorders, 4^th Edition (DSM-IV)1 criteria of any Axis-I disorder, (vi) not fulfilling DSM-IV criteria of current or lifetime alcohol and substance abuse, (vii) no current serious medical disorder, (viii) no current or life time personal history of neurological disorder or symptoms, (ix) no family history (first degree relatives) of any DSM-IV Axis-I disorder or hereditary neurological disorder, and (x) no prior familiarity with or knowledge of neurocognitive tests.

Volunteers were initially screened for suitability using a detailed questionnaire developed for this study. Those eligible were further assessed using the Mini International Neuropsychiatric Interview (MINI) screen9 and those endorsing any of the screen questions were excluded. The study was approved by the scientific and ethical committees of AIMS. Written informed consent was obtained from all participants.

USA comparison sample: This sample was drawn from a large database of healthy subjects from Hartford, Connecticut, held at the Olin Neuropsychiatry Research Center that has been used in the development of JANET. The absence of any personal history of psychiatric disorders in this group was based on the Diagnostic Interview for Genetic Studies10. Subjects were further selected to match the Indian sample on sex, age and years of education.

Cognitive battery: The cognitive battery was drawn from the JANET (http://janet.glahngroup.org) computerized neuropsychological battery developed in the JAVA programming language that is not platform or operating system dependent. It provides a fully automated testing environment which logs subject information, administers tests in a pre-defined order and databases subject performance at the completion of each test. This study was focused on the six tests chosen because these are largely language independent and represent exemplars of tasks used for the evaluation of key cognitive domains:

1. Matrix reasoning11: This is a subtest of Wechsler Abbreviated Scale of Intelligence which assesses general intelligence and reasoning ability. Participants are shown a total of 35 patterns, each missing one part. They are asked to complete the pattern by selecting one of five parts displayed with each pattern. The test last on average 15 min or is terminated after 4 wrong responses out of the 5 consecutive answers. Performance was evaluated in terms of number of correct and incorrect responses.

2. The Identical Pairs Continuous Performance Test, 3-Symbol Version (IPCPTS)12: The IPCPTS measures sensory/motor processing speed, attention and working memory. In this test, sets of 3 different shapes are sequentially shown and participants are required to indicate when two identical sets are sequentially presented. In total 70 sets are presented over an average run time of 7 min. Performance was evaluated in terms of accuracy (number of correct hits) and errors of omission (number of misses).

3. Spatial Capacity Delayed Response Test (SCAP)13: This test assesses spatial delayed working memory. Participants are shown sets of 3 or 5 yellow dots spaced randomly on the computer screen. After a few seconds they disappear and are replaced with a single green dot. Participants are required to indicate whether the green dot occupies the same spatial location of any of the yellow dots previously displayed. A total set of twelve patterns with 3 dots and twelve with 5 dots are presented with an average runtime of 5 min. Performance was evaluated in terms of the number of correct and incorrect responses and response time for each load and in total.

4. Stop Signal Reaction Time (SSRT)14: Stop Signal Reaction Time is a test of motor inhibition. Here the subject is shown some airplanes on the screen and they are required to push the right arrow button if the airplane is pointing to the right and the left arrow button if the airplane is pointing to the left. If a bomb appears just after the airplane the subject is required not to push any buttons. The subject is given examples. Airplanes minus the bombs appear 100 times and airplanes with the bomb appear 40 times and the average running time of the test is 4.91 min. Performance was evaluated in terms of the number correct and incorrect go and stop trials and response time to correct stop trials.

5. Emotional Recognition15: This is a test of emotional judgement capacity. Participants are shown 40 facial identities, one at a time, each depicting one of 5 emotional states namely happiness, sadness, anger, fear and neutral. Across emotional categories, stimuli are balanced for gender and ethnicity. For each facial identity participants are asked to choose from the list of the above emotions the one that matches the emotion expressed. Average testing time is 4 min. Performance was evaluated in terms of the number of male and female facial expressions correctly identified.

Balloon Analogue Risk Task (BART)16: The BART is a test of risk taking behaviour. During the BART, participants are shown on the computer screen a simulated balloon and a balloon pump. Also displayed is a stop function and a display of the points earned in total as well as the points earned during the current balloon. Participants are instructed to press the pump function to inflate the balloon but to be mindful that the balloon may blow up at any point even as early as the first pump press. They are also told that they will earn one point each time they use the pump to inflate the balloon but they will lose all their points if the balloon pops. At any point they have the option of pressing the stop function to collect all the points from that balloon. Each balloon trial ends either when the balloon blows up or when the participant chooses the stop function. There are 15 trials in total and the run time of the test is 4 min on average. Performance was evaluated in terms of the number of points won for successful trials, number of unsuccessful trials, number of points for unsuccessful trials and number of points after successful and after unsuccessful trials.

Data acquisition in both sites was supervised by trained research psychologists. For the Indian sample, the standardised test instructions were translated in Malayalam. Tests were administered in the same order as listed above in a single session.

Statistical analysis: The statistical analyses were performed using the Statistical package for Social Sciences (SPSS), version 15 (www.spss.com). The normality of data distribution for each outcome variable was tested using the Kolmogorov-Smirnoff test. General linear models were used to determine the unadjusted and independent effects of age, sex and education on cognitive test scores. A similar approach was used to estimate the proportion of variance explained by site.

Results

Indian sample: A total of 133 individuals were screened for eligibility. Fourteen individuals failed the M.I.N.I. screen as they responded positively to screen questions for anxiety (n=5), eating (n=1) and mood disorders (n=8), nine individuals were excluded because of medical morbidity and 10 withdrew their consent.

The total sample comprised 100 individuals. The mean age (per decade), sex and years of education of participants are shown in Table I. Data on the IPCPTS from four subjects and from one subject on the BART were unavailable for technical reasons. Tables II–VII show the descriptive statistics of the key outcomes from each test by decade and sex; for normally distributed variables mean and standard deviation and presented and for variables that did not have a normal distribution, median and Interquartile range are given. Table VIII summarises the independent effects of age, sex and education for each cognitive outcome variable.

Table I Demographic characteristics of the Indian (n=100) and USA sample (n=85)

Table II Performance on Matrix Reasoning by age and gender in the Indian (n=100) and USA sample (n=85)

Table III Performance on Identical Pairs Continuous Performance Test - 3 Symbol Version (IPCPTS) by age and gender in the Indian (n=100) and USA sample (n=85)

Table IV Performance on Spatial Capacity Delayed Response Test (SCAP) by age and gender in the Indian (n=100) and USA sample (n=85)

Table V Performance on Stop Signal Reaction time (SSRT) by age and gender (n=100)

Table VI Performance on Emotion Recognition Test by age and gender (n=100)

Table VII Performance on Balloon Analogue Risk Test (BART) for the Indian (n=100) and USA sample (n=85)

Table VIII The independent effects of age, sex and education

In general the effect of sex was minimal. However, since an effect of sex has been observed in the BART17 in healthy 18-25 yr olds we examined the effect of sex for this task in each age group separately. We found an effect of sex restricted to 18-27 yr olds where it explained 5 per cent of the variance in performance both in terms of failed trials and number of total points earned, the first favouring men and the latter favouring women.

Participants showed near ceiling levels of accuracy during affect recognition for happy and sad faces (regardless of the gender of the facial identity). Compared to happy and sad faces, accuracy was significantly (P<0.001) lower for fearful, angry and neutral faces without further differences between these three facial expressions.

US sample: A total of 85 healthy individuals were selected from the JANET database to match the Indian sample on sex, age and years of education (Table I).

In the tests where comparable data were available, the performance of the two groups was nearly identical (Tables II, III, IV and VII) and site contributed less than 0.6 per cent of the variance for any of the outcome measures.

Discussion

Comparison between the Indian and US sample showed that performance on the tests included in this battery was minimally affected by site, thus supporting the potential of trans-cultural research including sites in the Indian subcontinent. In the Kerala sample, the effect of sex on all cognitive test outcome variables was minimal and not significant. However, performance on the BART showed a significant effect of gender restricted in the 18-27 yr age group. Previous studies on the effect of sex on BART performance have been inconsistent16–18 but suggest that this issue requires further investigation. At the same time, overall performance in this test was comparable to that of the USA sample, aligned with previous reports from western countries and appears independent of education16–18. These observations favour the use of BART as a measure of risk taking behaviour in cross-cultural studies.

Participants’ accuracy in the Emotion Recognition task was generally high, particularly for happiness and sadness and less so for anger and fear. These findings are consistent with results from multiple studies examining facial affect recognition in individuals from diverse cultural backgrounds19–21 and appear more encouraging than previous reports of lower performance in this task in healthy individuals of Indian origin22.

In the Kerala sample, increasing age was associated with longer reaction times in the SCAP and SSRT accounting for 4.4 and 2.1 per cent of the variance and worse performance in Matrix Reasoning, IPCPTS and SSRT. In contrast, higher educational level was associated with better performance on Matrix Reasoning, IPCPTS and SCAP accounting between approximately 3-5 per cent of the variance. The influence of age and education was therefore, consistent in size and direction with that reported in investigations from western countries and comparable to that of the USA sample.

Certain methodological limitations of this study suggest avenues for future research. Firstly, the sample was restricted to individuals who were literate and had completed basic primary and secondary education. The issue of literacy is not a key concern in western countries but this is not the case for low income countries. The State of Kerala in India has the highest literacy rates averaging 96.02 per cent for males and 91.98 per cent for females according to official State statistics23. We used subtests of the JANET that were language independent to allow comparison with studies from other countries. However, verbal comprehension, expression and learning are important dimensions of cognitive function often implicated in severe mental illness2425. Therefore, future studies will need to focus on developing culturally equivalent language based tasks.

Following current concepts for cross-cultural standardization26 we addressed the issues of (i) functional equivalence (adequacy of translation) by choosing language independent tasks and translating standardised instruction, (ii) operational equivalence (standardization of psychometric testing procedures) by choosing a rater independent computerized test battery, and (iii) we have demonstrated the metric equivalence (transferability of scoring results from one culture to another) of neurocognitive tests of general intellectual ability, attention, working memory, response inhibition, emotional recognition and risk taking in healthy individuals from south India.