Use of EEfRT in the NIH study: Deep phenotyping of PI-ME/CFS, 2024, Walitt et al

I think it is worth while to look at issues at all levels. I tend to focus on the broad brush but I think the detail is equally important.

One simple thing I am wondering about:
The paper proposes the general interpretation that peripheral tissues are essentially normal and that the problem is that central signals either prevent activities or make them seriously unpleasant. That to me is a reasonable idea but Nath's comments suggest confusion between two different types of central signalling.

In flu, signals due to cytokines either make actions impossible (you actually fall over if you try to stand) or make actions induce unpleasant symptoms like nausea or 'fatigue'. My understanding is that in ME the situation is similar, in both forms, except that whereas in flu unpleasant symptoms occur within a couple of minutes (plus maybe more payback after an hour or so) in ME the reaction is delayed and can be very long lasting.

What the effort tests test for, however, is either unconscious conditioned responses or deliberate conscious responses, acquired because of experiencing the 'flu signals'. These responses would constitute a second and quite different set of signals. (There may or may not be valid evidence for such responses.)

We have two sets of signals we might call flu signals and pacing signals. What is blindingly obvious with flu and Covid is that flu signals cause the pacing signals. But suggesting effort task data provide an explanation implies that pacing signals are responsible for flu signals, or just are flu signals, which is unjustified.

Put differently, whether or not PWME make different effort choices provides no explanation for being disabled.
 
Eddie said:
The true optimal strategy would depend on how the rewards are generated (are they randomly selected?, are they on a bell curve?) but it certainly involves failing the lower reward tasks on purpose.
Click to expand...
As far as I've seen the generation of the reward money is purposely not told to players so that optimal strategies are harder to find (this was written in a different EEfRT paper), furthermore I've never seen the distribution specified in any of the papers. In this case a truely optimal strategy would envolve estimating this distribution on the space of all possible distributions on ([1.24,4.30]), whilst also accounting for multiple other things (estimating your own abilities, what kind of rewards have already been paid out to you, what combinations have already appeared etc). The complexity of this will be far too high for any person playing the game. Something that the participants might know is that "Each level of probability appeared once in conjunction with each level of reward value for the hard task." (I believe this should probably say "at least once" and I would assume level refers to the 3 levels in to which the rewards are typically categorised), so participants can make some good decisions based just on that, as long as they have a good memory and have fully understood everything (which most pwME probably won't have in this setup).

However, the examiners and this is what other studies have done as well, might still exclude data from outliers that use "certain" strategies, especially those focused on maximising pay-out, by only focusing on completing hard and high pay out tasks, if even if they are not truely optimal in the classical sense. In this case the details start becoming important. How exactly and in which setting were the rules explained to the participants and on what kind of basis were exclusions made (pre-defined rules, post-hoc analysis focused only on one part of the analysis), this includes looking at whether other participants were not excluded even though they could have been based on different rules (for example what the one study that @Murph cited calls "non-systemic responding"), which in this case could imply an exclusion bias.
 
Last edited:
Another simple point is that if an experiment reveals the cause of some pathological state, you expect to see more or less complete separation of populations. If it really is the single cause, which is quite often identifiable, the separation must be complete.

Correlations and slopes and box plots and p values are interesting if you are just looking for a clue as to what sort of cause may be involved. But the interpretation in this paper seems to be that a cause has been found. Nothing of the sort.
 
Jonathan Edwards said:
Another simple point is that if an experiment reveals the cause of some pathological state, you expect to see more or less complete separation of populations. If it really is the single cause, which is quite often identifiable, the separation must be complete.

Correlations and slopes and box plots and p values are interesting if you are just looking for a clue as to what sort of cause may be involved. But the interpretation in this paper seems to be that a cause has been found. Nothing of the sort.
Click to expand...

Whilst this might be correct argument from a biomedical stand-point and should be more clear to researchers and reviewers of journals in general, I fear it might be harder to make this argument in the context of responding to the EEfRT part of the study. That is because the results of the EEfRT seem to be somewhat unrobust in all kinds of different trials and yet others have often drawn some rather strong conclusions themselves, so Walitt et al might be able argue that their drawing of conclusions is consistent with the literature, even if the literature itself is inconsistent.

I currently feel like focusing on the "unrobustness" of the results in combination with an wrongful application of the test could be a sufficiently strong argument, without having to get into a debate on whether any of the results, which these authors and many others have often presented, have ever meant anything to begin with.
 
EndME said:
That is because the results of the EEfRT seem to be somewhat unrobust in all kinds of different trials and yet others have often drawn some rather strong conclusions themselves, so Walitt et al might be able argue that their drawing of conclusions is consistent with the literature, even if the literature itself is inconsistent.
Click to expand...

Yes, I was not particularly suggesting using this as an argument in a critique. It was more just suggesting reminding ourselves how far away from any rational argument their interpretation of data like this is.
 
Have I got this right?

Surely highlighting in the abstract any hypothesis where it's support depends entirely on a single tiny study using a test not validated for that disease, where the p value only just falls below the already rather generous 0.05 level, and where the exclusion or inclusion of a single data point makes the difference between going below or above p=0.05 should be thrown out by reviewers of the paper.
 
Jonathan Edwards said:
I think it is worth while to look at issues at all levels. I tend to focus on the broad brush but I think the detail is equally important.

One simple thing I am wondering about:
The paper proposes the general interpretation that peripheral tissues are essentially normal and that the problem is that central signals either prevent activities or make them seriously unpleasant. That to me is a reasonable idea but Nath's comments suggest confusion between two different types of central signalling.

In flu, signals due to cytokines either make actions impossible (you actually fall over if you try to stand) or make actions induce unpleasant symptoms like nausea or 'fatigue'. My understanding is that in ME the situation is similar, in both forms, except that whereas in flu unpleasant symptoms occur within a couple of minutes (plus maybe more payback after an hour or so) in ME the reaction is delayed and can be very long lasting.

What the effort tests test for, however, is either unconscious conditioned responses or deliberate conscious responses, acquired because of experiencing the 'flu signals'. These responses would constitute a second and quite different set of signals. (There may or may not be valid evidence for such responses.)

We have two sets of signals we might call flu signals and pacing signals. What is blindingly obvious with flu and Covid is that flu signals cause the pacing signals. But suggesting effort task data provide an explanation implies that pacing signals are responsible for flu signals, or just are flu signals, which is unjustified.

Put differently, whether or not PWME make different effort choices provides no explanation for being disabled.
Click to expand...

"My understanding is that in ME the situation is similar, in both forms, except that whereas in flu unpleasant symptoms occur within a couple of minutes (plus maybe more payback after an hour or so) in ME the reaction is delayed and can be very long lasting."

I don't think the distinction is as clear cut as that. For many of us, fatiguability effects will start to build as soon as the activity is started, with the combined effects of fatiguability on concentration and on the physical exertion of sitting, holding the arm in position and finger movement. It's likely that, for many pwME, symptoms like pain, nausea, slowed concentration, muscle fatigue etc will build during the activity, not just anticipation of PEM later.

In this situation where the pwME are likely to be already building up to or in rolling PEM through the multlple tests they are subjected to from other parts of the study on top of travel to get there etc, I think it's highly likely they will be feeling crap at the start, and very crap by the end of the task.
 
Trish said:
It's likely that, for many pwME, symptoms like pain, nausea, slowed concentration, muscle fatigue etc will build during the activity, not just anticipation of PEM later.
Click to expand...

I agree. During activity started from a well rested state, my fatiguability symptoms are similar to those of 'flu, but aren't as severe at the outset.

Activity attempted during a PEM state produces symptoms just like 'flu. The pattern includes that odd stage during recovery, where an activity feels possible whilst at rest but proves completely unfeasible as soon as it's attempted.
 
I think I’m not unusual in having both fatiguability and PEM separated by time.

I know some researchers like to roll both concepts into one. But they are absolutely not the same!

The fatiguability is physically limiting right then in the moment, but it is the delayed PEM that concerns me more regarding my ongoing well-being, and it’s that which I monitor for future planning. (IME Slow burn fatiguability is the most PEM inducing.)

So in terms of this finger tapping exercise, I think it very likely that some pwME will be physically unable to complete harder tasks due to the fatiguability.

The weighing up of the PEM is, of course, another whole layer of complexity for pwME to consider as the tests continue.

Very different to the situation for the HV’s who don’t have to deal with either issue.

The test is therefore not comparable between the 2 groups. At least not as a measure of “effort preference”!
 
Keela Too said:
I think I’m not unusual in having both fatiguability and PEM separated by time.

I know some researchers like to roll both concepts into one. But they are absolutely not the same!

The fatiguability is physically limiting right then in the moment, but it is the delayed PEM that concerns me more regarding my ongoing well-being, and it’s that which I monitor for future planning. (IME Slow burn fatiguability is the most PEM inducing.)

So in terms of this finger tapping exercise, I think it very likely that some pwME will be physically unable to complete harder tasks due to the fatiguability.

The weighing up of the PEM is, of course, another whole layer of complexity for pwME to consider as the tests continue.

Very different to the situation for the HV’s who don’t have to deal with either issue.

The test is therefore not comparable between the 2 groups. At least not as a measure of “effort preference”!
Click to expand...
precisely
 
As many of us have now said a bunch of different ways - its not an appropriate or fair comparison.

You cannot do a test for 'effort preference' when one set of subjects has a range of negative consequences to factor in when doing it (both in-real-time fatigability & the later 'payback' of PEM), and the other set has no negative consequences to factor in whatsoever.

Thats seems so obvious, it seems absurd to miss it :confused:

You cannot compare motivation or anything else when one set has only reward to deal with, and the other set have both "reward" & punishment. Its a game for one and a major life affecting set of calculations for the the other!
(and thats assuming the "+punishment" set are not too impaired to think clearly to calculate it in any case - which is unlikely!)

So the whole thing is just a complete joke AFAICS

(minor editing for sense & typos)
 
Jonathan Edwards said:
in ME the reaction is delayed and can be very long lasting.
Click to expand...
It has both immediate and delayed components.

Sometimes I go and do stuff and actually feel better afterwards. It feels to me just the normal response to moderate activity. Sometimes it is difficult and unpleasant to do, and hurts from start to finish, and I feel worse more or less immediately afterwards, and I know it will before I start. Usually it is somewhere in between.

Either way there will be a delayed component. A bill come due in a day or so. Obviously decades of practice has vastly improved my ability to minimise it, and optimise what activity I can do, as far as possible. But it is always there.

The delayed component (PEM is the key part). One thing that has changed for the better for us is that PEM is taken a lot more seriously. Still relatively early days for the profession in coming to grips with it, but at least they are listening more carefully and seriously now.

Plus what @Keela Too said, better than I did. :thumbsup:
 
Last edited:
What ever triggers immediate fatigue may contribute to subsequent PEM, and indeed this rapid fatiguability may be more noticeable or have more rapid onset if already in PEM, but I agree my subjective experience is the two are distinct.

If not already in PEM or experiencing concurrent ’rolling’ PEM, if I am experiencing rapid fatiguability immediately stopping the activity and resting will see the developing symptoms fair promptly dissipate, which is different to PEM which is delayed and can paradoxically continue to worsen even when resting. Also initial symptoms of rapid fatiguability are more likely to be directly related to the activity, such as muscle ache or tremor, where as the symptoms in PEM may include those related to the activity but also includes other not directly related symptoms.

I have similar experiences with sensory hypersensitivities, for example if a smell or a noise starts triggering adverse symptoms escaping the stimulus sees an immediate and reasonably prompt diminution in those symptoms, but persisting in that situation sees the symptoms worsen and can also lead to subsequent triggering of PEM.
 
Eddie said:
The researchers themselves don't seem to understand the optimal strategy to maximize winnings.
Click to expand...
They either don't understand that there is such a strategy, or didn't consider it relevant. The simple use of hard:easy ratio as the final outcome says it all, since doing the test as they intended it to be used doesn't maximize earnings, and that's discounting the fact that the earnings are so pitiful that the real motivation isn't about rewards.

That Nath replied that using the optimal strategy is "not following instructions" says it all, when in fact this tester is the only participant who did it the right way to maximize their earnings, the only player to actually play the game, rather than do what the researchers expected.

"Don't hate the player, hate the game" was invented for that.
 
You must log in or register to reply here.
Back
Top Bottom