Are objective outcomes of cognitive function possible?

[Inara]

I have a question concerning objective outcomes. Please note I don't mean to not use objective outcomes.

How would you measure cognitive function objectively?

E.g. my cognitive function improved, but physical function seems to be unchanged. Personally, I value a better cognitive function very much (although I would be glad everything would be ok) - would it be much better I could probably start working again - so I wouldn't say it's a worthless parameter.

But how to measure such parameters objectively?

Did the PACE authors look at cognitive function, and how did they measure it?

 

[Barry]

I have a question concerning objective outcomes. Please note I don't mean to not use objective outcomes.

How would you measure cognitive function objectively?

E.g. my cognitive function improved, but physical function seems to be unchanged. Personally, I value a better cognitive function very much (although I would be glad everything would be ok) - would it be much better I could probably start working again - so I wouldn't say it's a worthless parameter.

But how to measure such parameters objectively?

Did the PACE authors look at cognitive function, and how did they measure it?

This has to be a question for our scientifically inclined members. I do know that quite a lot of things cannot be measured objectively, being as so many salient outcomes are themselves subjective. Which means you then need blinding in a trial somehow, but if that is not possible then I don't know how you go about resolving that problem. Any thoughts on this @Jonathan Edwards?

Edit: The PACE approach is of course unacceptable - to just say it's too difficult a problem to resolve, so ignore it and presume the results will be OK anyway.

 

[Art Vandelay]

How would you measure cognitive function objectively?

E.g. my cognitive function improved, but physical function seems to be unchanged. Personally, I value a better cognitive function very much (although I would be glad everything would be ok) - would it be much better I could probably start working again - so I wouldn't say it's a worthless parameter.

But how to measure such parameters objectively?

Many years ago, I participated in a PhD student's study that attempted to test cognitive function in CFS so my memory of it is pretty hazy. It involved a lot of games, tests and puzzles with a focus on short term memory I think. They were fairly basic tests that you might give to a primary school student.

I remember doing really well at it (which was a shock to both me and the student I believe) but I crashed for about a week afterwards due to the mental effort. It's unfortunate that she didn't see this consequence of it!

 

[Jonathan Edwards]

How would you measure cognitive function objectively?

That is a very complicated question. As Art says you could measure ability at puzzles or maths tests. But that could be affected by motivation and belief. Art might have said half way through ' I am not going to do any more of this because last time I did I crashed for a week'.

In rheumatoid arthritis we have a series of measures that range in objectivity from complete (blood tests) to not at all (pain score). In between there are measures that may be influenced to some degree by attitudes of either patient or assessor (swollen joint count). We do not try to define objectivity but rather make use of a combination of objective and subjective measures. That allows us to see if the endpoints that really matter to patients (often subjective) change in parallel with more objective findings that make it likely that the change really reflects disease change. If we need something as complicated as that for RA I think it is almost certain that we need something that complicated for ME/CFS - agreed in advance.

 

[Peter T]

How would you measure cognitive function objectively?

There are a large number of potentially objective measures of cognitive function, though there would need to be a lot done to evaluate which were relevant to ME. There are general features that are impacted by most conditions such as short term memory problems or word finding difficulties, that you might expect every one with ME to have problems with at some time or other, but there are also more specfic problems that different people with ME report such as some report problems with visual patterns or one of my personal ones is losing of my sense of direction and navigation skills. We have no idea whether some specific problems are common to everyone with ME or if everyone experiences different difficulties.

Obviously there is the problem that each person is different, not everyone has the same premorbid cognitive skills, also you can not normally go back in time and test people before their ME started, so there would need to be very large scale studies if you were looking at group averages, however it would be very straight forward to look at changes within an individual. A good starting point would be a simple digit span test, asking people to repeat a string of numbers, this is the type of memory you use to dial a telephone number you do not know by heart, the average person is able to repeat 7 digits (plus or minus two). When I am exoeriencing post exertional malaise this is definitely impaired for me, I can only dial telephone numbers by having the number written in front of me and putting my finger on each number in turn.

One problem is getting to test people at the right time, you would need to catch subjects when experiencing post exertional malaise. So in the example @Art Vandelay gives, the student should have repeated the assessments over several days.

Also as @Art Vandelay mentions there are potential problems of ceiling effects, that is if the tests used are too easy so everyone scores full marks, for example if a task it is too simple it will not show up an individual's cognitive difficulties, though a harder test might do. There is also the comparable situation where a task is too hard so everyone fails it.

However all these problems would not be insurmountable for a competent cognitive neuropsychologist, unfortunately historically psychologists involved have allowed themselves to be sidetracked by the psychosocial model and it is not normal for neuropsychological assessments to be undertaken.

 

[Peter T]

Using cognitive tests that reflect general impaired function a single test would not distinguish between fatigue inducing conditions, for example with a digit span you would expect to see it impaired in people with depression as well as people with ME, however repeated testing may reveal interesting changes. If you used a walking exercise, this would not be expected to have an negative effect on subsequent retesting with some one with depression, it could even result in an improvement whereas you would expect a subsequent negative effect in those with ME experiencing post exertional malaise.

It is also possible to attempt subjective evaluation of specific tasks, which might be more reliable than a general question about how is your general intellectually functioning today. For example when I am lying in bed doing as little as possible I often play solitaire on my iPad. Though it is possible to get objective measures of speed to complete a game and frequency of 'winning', I suspect my subjective experience of how well my brain is functioning in relation to this specific task is much more accurate than my general assessment of my congnitive functioning. So if I am recording variation in activity and symptoms, I would rely more on my performance in specific tasks like this to rate my cognitive function rather than a general rating of how well my brain seems to be functioning.

 

[Peter T]

Sorry I meant to add that I seem to remember that some of the research on post exertional malaise has looked at some simple tests of cognitive function but can not of the top of my head remember any details, it maybe that some readers with a more reliable episodic memory than me could fill in the gaps.

 

[Inara]

Thank you all for explaining very clearly. I see there are enough ways to assess cognitive function. I think a reasonable approach would be the one @Jonathan Edwards explained, plus using 'common sense' and thinking.

 

[Esther12]

How would you measure cognitive function objectively?

There was this attempt, from two of the same trial which had the null results for actometer data:


The effect of cognitive behaviour therapy for chronic fatigue syndrome on self-reported cognitive impairments and neuropsychological test performance
Hans Knoop, Judith B Prins, Maja Stulemeijer, Jos W M van der Meer, and Gijs Bleijenberg

Abstract

Background
Patients with chronic fatigue syndrome (CFS) often have concentration and memory problems. Neuropsychological test performance is impaired in at least a subgroup of patients with CFS. Cognitive behavioural therapy (CBT) for CFS leads to a reduction in fatigue and disabilities.
Aim
To test the hypothesis that CBT results in a reduction of self-reported cognitive impairment and in an improved neuropsychological test performance.
Methods
Data of two previous randomised controlled trials were used. One study compared CBT for adult patients with CFS, with two control conditions. The second study compared CBT for adolescent patients with a waiting list condition. Self-reported cognitive impairment was assessed with questionnaires. Information speed was measured with simple and choice reaction time tasks. Adults also completed the symbol digit?modalities task, a measure of complex attentional function.
Results
In both studies, the level of self?reported cognitive impairment decreased significantly more after CBT than in the control conditions. Neuropsychological test performance did not improve.
Conclusions
CBT leads to a reduction in self?reported cognitive impairment, but not to improved neuropsychological test performance. The findings of this study support the idea that the distorted perception of cognitive processes is more central to CFS than actual cognitive performance.

http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2077788/

They don't actually give us the results from the tests though, to let us see how they compare to healthy controls. They talk as if 'CBT is great because these patients are too worried about neuropsychological problems', but don't address how serious those problems actually are, and how concerned about them it would be reasonable to be! (I previously said that the paper isn't really worth reading - can't remember it now though).

These are the tests they use:

Neuropsychological tests

Reaction time task
The reaction time task consisted of two subtests, simple and choice reaction time tasks. Both are described in detail elsewhere.8,15 In a previous study, the reaction times of patients with CFS were slower than that of healthy controls on both tasks.8

Symbol digit modalities task

The symbol digit modalities task (SDMT)16 was used in the adult study as a measure of complex attention. In previous studies, patients with CFS scored lower than a matched healthy control group.8,9

It's another example of CBT being great at getting patients to answer questionnaires in a more 'healthy' manner, but not so good at actually reducing disability. I get the impression that for a lot of doctors, they mainly want CFS patients to complain less... that's their priority for any treatment!

 

[Inara]

In both studies, the level of self?reported cognitive impairment decreased significantly more after CBT than in the control conditions. Neuropsychological test performance did not improve.

Well, my impression is, here again: subjective outcomes probably show improvement, objective ones don't. Or differently: CBT is a good way to make you believe whatever you're supposed to believe.

 

[Inara]

The findings of this study support the idea that the distorted perception of cognitive processes is more central to CFS than actual cognitive performance.

Interesting conclusion. Instead of concluding that CBT doesn't objectively improve cognitive function, it seems they conclude that patients imagine their cognitive impairments.

 

[Inara]

I get the impression that for a lot of doctors, they mainly want CFS patients to complain less... that's their priority for any treatment!

Yes.

 

[Barry]

Well, my impression is, here again: subjective outcomes probably show improvement, objective ones don't. Or differently: CBT is a good way to make you believe whatever you're supposed to believe.

Interesting conclusion. Instead of concluding that CBT doesn't objectively improve cognitive function, it seems they conclude that patients imagine their cognitive impairments.

Beat me to it. The authors have a distorted perception of what improvement and science is. What an astonishing conclusion!

 

[sea]

CBT leads to a reduction in self?reported cognitive impairment, but not to improved neuropsychological test performance. The findings of this study support the idea that the distorted perception of cognitive processes is more central to CFS than actual cognitive performance.

Interesting conclusion. Instead of concluding that CBT doesn't objectively improve cognitive function, it seems they conclude that patients imagine their cognitive impairments.

Once again it's where we need objective markers. Of course it's important whether or not a treatment makes a patient feel better - but that isn't the primary goal in treating illness. Imagine if chemotherapy was appraised by how it made patients feel rather than on what it achieved in stopping cancer. How many people with things like diabetes or heart disease stop taking various medications because they "feel better" but their actual condition is still serious.

 

[Woolie]

I have a question concerning objective outcomes. Please note I don't mean to not use objective outcomes.

How would you measure cognitive function objectively?

E.g. my cognitive function improved, but physical function seems to be unchanged. Personally, I value a better cognitive function very much (although I would be glad everything would be ok) - would it be much better I could probably start working again - so I wouldn't say it's a worthless parameter.

But how to measure such parameters objectively?

Did the PACE authors look at cognitive function, and how did they measure it?

Good question. Objective to me means something that does not involve people having to judge for themselves how they think their function is. So any measure that doesn't involve self-evaluation is a great start.

Step 1 is to pick a task which taxes people cognitively in the right kind of way. I suspect in MECFS, this would involve some effortful task, where that effort must be sustained for a reasonable period. If I wanted something really sensitive, I would maybe pick some sort of dual task paradigm, like listening for a particular word while at the same time writing to dictation. Another option would be some selective attention task where the distractors (the things you're not supposed to respond to) are very similar to the things you are supposed to respond to.

It would be important to have some task which is continuous for a good amount of time, like 20 minutes.

I'd measure speed as well as accuracy.

This is all just a guess, and you'd need to spend some time trying different tasks to see which ones are sensitive, before using it as a measure of change.

 

[Woolie]

Reaction time task
The reaction time task consisted of two subtests, simple and choice reaction time tasks. Both are described in detail elsewhere.8,15 In a previous study, the reaction times of patients with CFS were slower than that of healthy controls on both tasks.8

This is not a great task. You want something that involves more complexity imo. Anything that just measures slowed responding is too general, won't be a specific enough marker of CFS-related cognitive dysfunction.

Symbol digit modalities task

The symbol digit modalities task (SDMT)16 was used in the adult study as a measure of complex attention. In previous studies, patients with CFS scored lower than a matched healthy control group.8,9

This is quite a good example of a continuous selective attention task.

 

[Inara]

If I wanted something really sensitive, I would maybe pick some sort of dual task paradigm, like listening for a particular word while at the same time writing to dictation. Another option would be some selective attention task where the distractors (the things you're not supposed to respond to) are very similar to the things you are supposed to respond to.

It would be important to have some task which is continuous for a good amount of time, like 20 minutes.

I'd measure speed as well as accuracy.

I think these are brilliant suggestions.

I assume that many people with ME can't multitask (anymore; I was great at multitasking once), and speed as well as accuracy are most probably a problem, too. And it's objectively measurable.

 

[Trish]

Lots of healthy people can't multitask. The main thing with a test involving multitasking I think, is comparing you with yourself, not with other people.

For example is there a difference in how well you do the test if you repeat it before you've had time to 'recover' your concentration from the first one. And do healthy people show the same pattern of change between repeated tests. A bit like the 2 day CPET, but doing a mental task instead of a physical one.

Does that make sense, @Woolie?

 

[Woolie]

Lots of healthy people can't multitask. The main thing with a test involving multitasking I think, is comparing you with yourself, not with other people.

For example is there a difference in how well you do the test if you repeat it before you've had time to 'recover' your concentration from the first one. And do healthy people show the same pattern of change between repeated tests. A bit like the 2 day CPET, but doing a mental task instead of a physical one.

Does that make sense, @Woolie?

Yea, all these more challenging cognitive tasks will have massive interindividual variability. The best you can do is find one that reasonably discriminates patients from the best possible matched controls, and then, as you say, you use it to measure change within individuals.

 

[Woolie]

There are a large number of potentially objective measures of cognitive function, though there would need to be a lot done to evaluate which were relevant to ME. There are general features that are impacted by most conditions such as short term memory problems or word finding difficulties, that you might expect every one with ME to have problems with at some time or other, but there are also more specfic problems that different people with ME report such as some report problems with visual patterns or one of my personal ones is losing of my sense of direction and navigation skills. We have no idea whether some specific problems are common to everyone with ME or if everyone experiences different difficulties.

Obviously there is the problem that each person is different, not everyone has the same premorbid cognitive skills, also you can not normally go back in time and test people before their ME started, so there would need to be very large scale studies if you were looking at group averages, however it would be very straight forward to look at changes within an individual. A good starting point would be a simple digit span test, asking people to repeat a string of numbers, this is the type of memory you use to dial a telephone number you do not know by heart, the average person is able to repeat 7 digits (plus or minus two). When I am experiencing post exertional malaise this is definitely impaired for me, I can only dial telephone numbers by having the number written in front of me and putting my finger on each number in turn.

Good suggestions, @Peter Trewhitt. I'm not sure about visual processing, spatial memory or short-term verbal memory - are these major features of ME related cognitive impairment? (I need to ask people because I don't get much brain fog myself). Up to now, I've been keen on the idea that ME impairs the ability to effectively utilise superior medial cortex to allocate resources for effortful tasks. So I've been thinking about effortful, demanding tasks.

What else do people experience when they have severe cognitive symptoms?