How citation distortions create unfounded authority: analysis of a citation network (2009) S A Greenberg

[Luther Blissett]

I think people will be interested in this paper which shows a way of detecting and showing how citation bias can create unfounded authority. The parallels with the whole BPS network seem striking. I've often seen reference to how certain authors cite themselves.

Abstract

Objective

To understand belief in a specific scientific claim by studying the pattern of citations among papers stating it.

Design

A complete citation network was constructed from all PubMed indexed English literature papers addressing the belief that β amyloid, a protein accumulated in the brain in Alzheimer’s disease, is produced by and injures skeletal muscle of patients with inclusion body myositis. Social network theory and graph theory were used to analyse this network.

Results

The network contained 242 papers and 675 citations addressing the belief, with 220 553 citation paths supporting it. Unfounded authority was established by citation bias against papers that refuted or weakened the belief; amplification, the marked expansion of the belief system by papers presenting no data addressing it; and forms of invention such as the conversion of hypothesis into fact through citation alone. Extension of this network into text within grants funded by the National Institutes of Health and obtained through the Freedom of Information Act showed the same phenomena present and sometimes used to justify requests for funding.

(emphasis added)

Conclusion

Citation is both an impartial scholarly method and a powerful form of social communication. Through distortions in its social use that include bias, amplification, and invention, citation can be used to generate information cascades resulting in unfounded authority of claims. Construction and analysis of a claim specific citation network may clarify the nature of a published belief system and expose distorted methods of social citation.

https://www.bmj.com/content/339/bmj.b2680

 

[Barry]

Having read your book @Brian Hughes, I'm guessing this might interest you, if you are not already aware of it. (I've not gone back to check if you reference this paper or not in the book, apologies if you did).

 

[Luther Blissett]

I forgot to add it, but this seemed to me the most interesting finding

Between 1996 and 2007 support for the claim grew exponentially, with the number of supportive citations and citation paths increasing sevenfold and 777-fold, to 636 citations and 220 553 citation paths. In contrast, the critical view grew to only 21 citations and 28 citation paths (fig 4⇓). No papers refuted or critiqued the critical data, but instead the data were just ignored. Analysis of a claim specific citation network can identify exactly which papers and citations have been most influential in pushing forward belief304 305 (see web extra note 7). The increased support was facilitated by a small number of papers, not reporting any primary data, through which large amounts of traffic (citation paths) flow in the network. For example, 63% of all citation paths (n=139 391) flow through one review paper21 (compared with 2% of citation paths flowing through randomly selected other papers); 95% of all citation paths flow through four review papers16 18 21 37 by the same research group (8% through four randomly selected other papers).

This was the part that really struck me on original reading as being similar to our situation.

 

[Jonathan Edwards]

The is quite cool @Luther Blissett. A bit like the inverted funnel plot - you may not be able to show one paper is crap but the whole load of papers reveals, without knowing it, a tell tale whiff of jiggery-pokery.

 

[inox]

So this is "how alzheimer research got it wrong for 20 years"...? *sigh*

There's no self correcting in science, if one ignores the evidence correcting you... Sounds way to familiar. :-/

 

[NelliePledge]

@dave30th in case you’ve not seen this thread

 

[Joel]

Cool. Would love to see such an analysis of CFS research.

 

[Barry]

Puts me in mind of The Lord of the Flies. With no well grounded datum, moralities tend to float and evolve according to group inclinations.

 

[JaimeS]

Cool. Would love to see such an analysis of CFS research.

Saaammme.

Though I expect we'd see it on the biomedical side, too, though not half so glaringly.

The effect of having a small sample size of studies to choose from is probably notable.