File Under "Cooking the Data"
Heavens! JAMA published a study suggesting that there may be some shenanigans in the way scientific papers are cited by those with an ax to grind. The article implies that "true believers" may be, shall we say...selective in their arguments when promoting their pet theories.
Click here for more.This is demoralizing. JAMA published a study suggesting that there may be some shenanigans in the way scientific papers are cited by those with an ax to grind. This article implies that "true believers" may be, shall we say...selective in their arguments when promoting their pet theories.
The authors looked at two observational studies published by the New England Journal of Medicine in 1993 suggesting that high vitamin E intake was associated with marked reductions in cardiac events. In 2000, an RCT was performed that showed no benefit (and possibly some deleterious effects). They then looked at how frequently the two "disproven" studies were cited later in the medical literature.
One would have expected a rapid fall-off in the citation curve after the RCT. In fact there was a fall-off but not nearly as precipitous as one might have expected. Even as late as 2005 (the last year they looked) 50% of the papers citing the original articles were still favorable towards vitamin E. Many authors were persisting in promoting vitamin A on the basis of two non-RCT's despite markedly better contradictory evidence that became available later.
This is rather damning against either the thoroughness of the citing authors (who perhaps missed the RCT in their literature searches) or against their honesty (in perhaps ignoring and not citing it). The researchers of the JAMA article did something rather slick that suggests dishonesty.
They looked at a possible correlation with how favorable towards vitamin E the citing articles were with whether or not they also cited the RCT. This is what's a bit disappointing. They found that 35% of the articles unfavorable or equivocal to vitamin E failed to cite the RCT but 83% of the pro-vitamin E failed to. Of course the erroneous conclusions could be because of sloppy research but it seems much more likely that true believers simply choose to ignore data that doesn't support their beliefs.
As my five-year-old likes to say when she doesn't get her way: "How sad".
Click here for more.This is demoralizing. JAMA published a study suggesting that there may be some shenanigans in the way scientific papers are cited by those with an ax to grind. This article implies that "true believers" may be, shall we say...selective in their arguments when promoting their pet theories.
The authors looked at two observational studies published by the New England Journal of Medicine in 1993 suggesting that high vitamin E intake was associated with marked reductions in cardiac events. In 2000, an RCT was performed that showed no benefit (and possibly some deleterious effects). They then looked at how frequently the two "disproven" studies were cited later in the medical literature.
One would have expected a rapid fall-off in the citation curve after the RCT. In fact there was a fall-off but not nearly as precipitous as one might have expected. Even as late as 2005 (the last year they looked) 50% of the papers citing the original articles were still favorable towards vitamin E. Many authors were persisting in promoting vitamin A on the basis of two non-RCT's despite markedly better contradictory evidence that became available later.
This is rather damning against either the thoroughness of the citing authors (who perhaps missed the RCT in their literature searches) or against their honesty (in perhaps ignoring and not citing it). The researchers of the JAMA article did something rather slick that suggests dishonesty.
They looked at a possible correlation with how favorable towards vitamin E the citing articles were with whether or not they also cited the RCT. This is what's a bit disappointing. They found that 35% of the articles unfavorable or equivocal to vitamin E failed to cite the RCT but 83% of the pro-vitamin E failed to. Of course the erroneous conclusions could be because of sloppy research but it seems much more likely that true believers simply choose to ignore data that doesn't support their beliefs.
As my five-year-old likes to say when she doesn't get her way: "How sad".
Labels: Junk Science, Statistics, Study Design
posted by The Medicine Man at 6:12 AM
0 Comments:
Post a Comment
<< Home