Feedback on underperformance in patients with CFS: The impact on subsequent neuropsychological test performance, 2018, Roor, Knoop et al

[Andy]

My interpretation: "If we tell people how they should have answered then they answer more correctly the second time around", or have I misunderstood something here?

Abstract
Performance Validity Tests (PVTs) are used to measure the credibility of neuropsychological test results. Until now, however, a minimal amount is known about the effects of feedback upon noncredible results (i.e., underperformance) on subsequent neuropsychological test performance.

The purpose of this study was to investigate the effects of feedback on underperformance in Chronic Fatigue Syndrome (CFS) patients. A subset of these patients received feedback on Amsterdam Short-Term Memory (ASTM) failure (i.e., feedback [FB] group). After matching, the final sample consisted of two comparable groups (i.e., FB and No FB; both n = 33). At baseline and follow-up assessment, the patients completed the ASTM and two measurements of information processing speed (Complex Reaction Time [CRT] and Symbol Digit Test [SDT]). Results indicated that the patients in the FB group improved significantly on the CRT, compared to the No FB group. Although not significant, a comparable trend-like effect was observed for the SDT. Independent of the feedback intervention there was a substantial improvement on ASTM performance at re-administration.

A limited feedback intervention upon underperformance in CFS patients may result in improvement on information processing speed performance. This implies that such an intervention might be clinically relevant, since it maximizes the potential of examining the patients’ actual level of cognitive abilities.

Open access at https://www.tandfonline.com/doi/full/10.1080/23279095.2018.1519509

 

[Esther12]

I think that the Amsterdam Short-Term Memory test is used to see if people are exaggerating memory problems, so this seems quite an odd study to me.

 

[rvallee]

I think that the Amsterdam Short-Term Memory test is used to see if people are exaggerating memory problems, so this seems quite an odd study to me.

Seems to be: https://www.ncbi.nlm.nih.gov/pubmed/9071640.

J Clin Exp Neuropsychol. 1997 Feb;19(1):43-51. Amsterdam Short-Term Memory test: a new procedure for the detection of feigned memory deficits

At least he's making his contempt quite clear. I wonder how he'll fare once his bullshit is exposed. Having built a career on nonsense can't be good for anyone. If only it mattered what impact it has on the patients though...

 

[Woolie]

Its amazing what you can get published in this area. It would never occur to me to submit this "study" for publication, because there's a flaw that undermines the entire thing: the feedback group repeated the tests one month later, whereas the non feedback group waited six months before re-testing.

Obviously, this is a major flaw, because the first group would benefit from practice effects, whereas the second would not.

I love how they try to make it sound like that major flaw can be overlooked, because it wasn't their fault! Sorry, but science doesn't care whose fault it was.

Before discussing the study findings in detail, we want to address one major limitation of this study: the difference in time interval between baseline and follow-up of the FB group (i.e., one month) and No FB group (i.e., six months). The difference in the time interval was caused by limited treatment capacity, and not by a systematic flaw in the study design. Also, the two groups were matched on baseline measures, and analyses showed that this matching was done without influencing outcome measurements. Nonetheless, the difference in time interval might have influenced our results. First to mention is the difference in practice effect (Heilbronner et al., 2010) in the two conditions. It could be argued that the shorter between-session period in the FB group resulted in higher scores at re-assessment compared to the No FB group. Unfortunately, the practice effects of the utilized neuropsychological tests in this study (i.e., SDT and CRT) for the two different time intervals in CFS patients are unknown. Additionally, the longer waitlist period in the No FB group may have resulted in changes in health status (i.e., worsening of symptoms) compared to the FB group, which could have affected the results.

 

[Hutan]

The difference in the time interval was caused by limited treatment capacity, and not by a systematic flaw in the study design.

Incredible. I find it very hard to believe that anyone could think this study was worth publishing.

 

[Cheshire]

the feedback group repeated the tests one month later, whereas the non feedback group waited six months before re-testing.

There seriously is something wrong with psychology, which, for the most part, has nothing to do with science.
But here, it is so obvious, I just can't believe it. It's not flaws anymore, it needs to be called what it is, it's f* fraud! And the way research in psychology is organised allows it.

@Woolie @Brian Hughes How is this possible?

 

[Hoopoe]

Once again, the Science For ME peer review almost immediately spots a fatal flaw.

 

[Hoopoe]

Also the implications here could be ugly: the test is designed to detect malingering; what kind of narratives would such a discrepancy support?

 

[Trish]

The researchers have made a conscious choice to interpret the results in the way that implies pwME don't try hard enough and can do better if they try.
In making this completely unjustified interpretation, the main thing they are doing is publishing for all to see, their preconceptions and prejudices about pwME.

 

[Sean]

Is this deliberate abuse of methodology, or just sheer ignorance and incompetence?

Regardless, how blatant does it have to get before the rest of medical science steps in and removes these arse clowns from the game?

 

[NelliePledge]

Sounds just about on the level of the shopping bag study