1. Guest, the 'News in Brief' for the week beginning 8th August 2022 is here.
    Dismiss Notice
  2. Welcome! To read the Core Purpose and Values of our forum, click here.
    Dismiss Notice

Response variability is associated with self-reported cognitive fatigue in multiple sclerosis, 2010, Bruce et al.

Discussion in 'Research methodology news and research' started by Woolie, Feb 3, 2020.

  1. Woolie

    Woolie Senior Member

    Messages:
    2,901
    I thought this piece of (oldish) research might be of interest to people interested in measuring cognitive dysfunction in ME. It notes that self reported fatigue in MS doesn't correlate well with actual cognitive performance measures (which is the same in ME). Here, they developed a new performance measure that appears to track self reports much better, which they call response time variability. Its a measure of the standard deviation of response times to items in a memory recognition tsk (excluding incorrect responses).

    full text of article can be downloaded here and here.
     
    Last edited: Feb 3, 2020
  2. ME/CFS Skeptic

    ME/CFS Skeptic Senior Member (Voting Rights)

    Messages:
    2,974
    Location:
    Belgium
    (haven't read the study yet) but why would response time variability have anything to do with fatigue if the response time itself isn't correlated. Sounds like a statistical correlation that doesn't mean much, to be honest.
     
  3. Woolie

    Woolie Senior Member

    Messages:
    2,901
    What they did is just create a performance measure from the standard deviation of the response times, and they found that high scores on this measure were associated with higher self-reported fatigue.

    The rationale seems pretty sound to me. The idea is that certain kinds of cognitive difficulties might exhibit themselves not in consistently slow performance, but in greater variability in performance (because small trial-to-trial changes in overall noise levels within the cognitive system may have abormally large effects on performance).
     
    Last edited: Feb 3, 2020
  4. Trish

    Trish Moderator Staff Member

    Messages:
    41,552
    Location:
    UK
    Neither of the links works for me and I couldn't get it on sci hub, so I haven't read the full article.

    I wonder whether the variability was actually a slowing down, or more erratic.
     
    alktipping likes this.
  5. Andy

    Andy Committee Member

    Messages:
    17,128
    Location:
    Hampshire, UK
    alktipping, MEMarge and Trish like this.
  6. Trish

    Trish Moderator Staff Member

    Messages:
    41,552
    Location:
    UK
    Thanks Andy. Don't know why it didn't work the first time I tried.

    To answer my own question, the discussion section suggests it's not just that fatigue leads to slowing down over a series of tasks, but that attention wanders and it's hard to get concentration back some of the time.

    From the discussion:
     
  7. Sid

    Sid Senior Member (Voting Rights)

    Messages:
    496
    Haven't read this paper yet but this is not a new concept. I recall reading a paper from 2007 or so about subjective cognitive complaints and increased standard deviation of performance on timed tasks in normal people. The lack of correlation between objective neuropsych tests and subjective cognitive problems is a perennial problem in research not just in ME but all kinds of neuro and psych conditions.
     
  8. Snow Leopard

    Snow Leopard Senior Member (Voting Rights)

    Messages:
    3,718
    Location:
    Australia
    One of the main problems with these studies is that they tend to be cross-sectional, so they don't have a conception of the pre-morbid performance of participants. Hence such studies are highly susceptible to participation biases, namely participants tend to have well above normal intelligence, skewing the results.

    In the case of healthy participants, the variation of the degree of impairment is probably insufficient compared to the easily-biased self-reports of subjective complaints and it is rarely ethical to subject healthy participants to whatever is required to cause significant impairment on such tests.
     
    MEMarge likes this.
  9. Sid

    Sid Senior Member (Voting Rights)

    Messages:
    496
    There are tests (reading and vocabulary) that can estimate premorbid intelligence pretty well so you can match your participants with controls that way. Commonly done in neuropsych.
     
    Sean and MEMarge like this.
  10. Snow Leopard

    Snow Leopard Senior Member (Voting Rights)

    Messages:
    3,718
    Location:
    Australia
    "pretty well"? How much of the variance does it explain with what samples? I suggest there is substantial variance with that method.
    There are many problems. What about individuals who became ill as teenagers and thus stunted their education? What about general differences in educational background and interests. Factors like this are compounded for individuals with significantly above average intelligence.
     
    Amw66 and Sean like this.
  11. Sid

    Sid Senior Member (Voting Rights)

    Messages:
    496
    You're not matching them for variance, you're matching them for mean premorbid intelligence.

    Yup, this is a big problem. We know that ME is the most common cause of long-term school absence and it's an illness that often strikes its victims in adolescence.
     
    MEMarge and Amw66 like this.
  12. Snow Leopard

    Snow Leopard Senior Member (Voting Rights)

    Messages:
    3,718
    Location:
    Australia
    Those who assume they can match them using such tests without understanding that such tests only explain some of the variance, and therefore do not fully predict premorbid intelligence are committing a deeper methodological sin...
     
  13. duncan

    duncan Senior Member (Voting Rights)

    Messages:
    1,165
    My personal experience with these is they can be inaccurate and subject to bias. For example, if I recall correctly, premorbid intelligence estimates are accompanied with confidence levels, eg, 90%. Well a 90% confidence is just plainly terrible.

    To make a long story short, in my n=1 story, two different psychs - hired by insurance companies on two separate occasions - missed my premorbid IQ by over 15 points.

    IMO, accordingly, those tests can suck. Moreover, it would appear to me, at least, some of those who execute them while on insurance company payroll may "estimate" in a manner that favors the insurance company.

    Patients test at their peril. Not that many have any choice.
     
    Last edited: Feb 16, 2020
    Mithriel and Snow Leopard like this.
  14. Snow Leopard

    Snow Leopard Senior Member (Voting Rights)

    Messages:
    3,718
    Location:
    Australia
    Exactly. Hypothetical example: premorbid IQ may be 130, but the test estimates 120 and current IQ test result is 120 due to poor concentration, brain fog etc. Yet they state there is no cognitive impairment because 120=120. Fact is that estimates of premorbid IQ have a greater error range the further one's IQ is from the mean.
     
    Mithriel and duncan like this.
  15. Sid

    Sid Senior Member (Voting Rights)

    Messages:
    496
    Yes, tests that estimate premorbid IQ have a low ceiling. Insurance companies have a vested interested in denying claims of cognitive impairment. Misuse of neuropsych testing in litigation cases or by pseudo-scientists such as BPS researchers does not mean that neuropsych testing can't be used appropriately, if you are trying to study the topic in good faith.

    Most of the population falls within +/- 2 SD so it would not be too difficult to design a study looking at a range of patients compared to normal people while excluding exceptionally bright people. 98% of people do not have an IQ of 130.
     
    Snow Leopard and duncan like this.
  16. duncan

    duncan Senior Member (Voting Rights)

    Messages:
    1,165
    Too many are not.

    I eviscerated my psych write-up precisely, in part, because the psych kept writing that my intelligence was "normal" - even as that psych kept repeating the mistake of underestimating my premorbid IQ by almost 20 pts. Do I think this was deliberate? I'm not really sure. I think this psych just found it easy to embrace using a low confidence level to mesh nicely with the position of the insurance carrier. No malice, perhaps, just an easy way out, at a patient's expense.

    Point is, the tests can be manipulated. As such, they can represent a danger. Until such a time as controls are tightened, eg confidence levels no less than 98%, etc, imo they should be disallowed.
     
    Last edited: Feb 17, 2020
    Sid and Snow Leopard like this.
  17. duncan

    duncan Senior Member (Voting Rights)

    Messages:
    1,165
    @Sid, I know what you are trying to say here, and I agree up to a point. However, what about those researchers/clinicians whose acts of good faith are finding results that support whatever team they happen to be behind? There are those who believe this is the greater truth.

    You see this same dilemma on the front lines of journalism.
     
    Last edited: Feb 17, 2020
  18. Snow Leopard

    Snow Leopard Senior Member (Voting Rights)

    Messages:
    3,718
    Location:
    Australia
    So excluding most s4me members then. ;)
     
    Mithriel and duncan like this.

Share This Page