Risks associated with over-analysis of data

Sir,

Chakraborty et al1 described a study on plasma adipocytokines and C-reactive protein levels in healthy school-going adolescents in Delhi. We wish to draw attention to risks associated with their over-analysis of data.

The authors did not state their primary outcome measures in either abstract or text; this encourages cherry-picking of statistically significant results2. Next, they conducted an analysis of data in which every variable was tested for association with almost every other variable in eight detailed tables; in fact, the analysis went to the extent of examining findings separately in lean and obese adolescents for each year of life between 12 and 18 yr. This was a data-dredging exercise, something that is discouraged because of the risk of Type 1 (false-positive) errors3.

Furthermore, the authors did not state that this was an exploratory study with no specific plan of analysis4,5, nor did they correct their analyses for a Type 1 error risk arising from multiplicity of hypothesis testing6. We, therefore, suggest that the significant findings that the authors presented be viewed with extreme caution because there is no way of knowing which finding is a result of a Type 1 error and which finding truly reflects what is true in the population.