Thanks, yes - I saw that presentation at the 2020 CMRC meeting, so that was my introduction to her work.
-
New Forum Software Installed (Still a few issues but reopening the forum) More details.
Independent advisory group for the full update of the Cochrane review on exercise therapy and ME/CFS (2020), led by Hilda Bastian
Yes. This is a political problem of why standards are not being followed for ME, and are indeed actively being diluted and discarded. It isn't a technical problem with the normal standards, that everybody else has to follow.
Last edited:
This.
We may not be able to adequately blind all measures in all situations, but we can use much more objective measures of the real-world practical consequences of the self-reported benefits, including the critical testing of correlations with self-report measures.
Patterns of changes in activity levels after therapy are certainly objectively measurable across a wide domain. Assuming, of course, that increasing those levels is the end point of therapy, which the highest of authority states is so:
We have very little information about Kay and specifically about her knowledge of ME/CFS issues, but the information that we do have suggests to me that she has not spent much time becoming familiar with the politics or the science of ME/CFS. I don't see how she can be effective in guiding the members of the review team who are not familiar with ME/CFS research.
I'm concerned about the process that Hilda applied in order to select the sole person with ME/CFS for the review team. There are many people with ME/CFS with good skills in research analysis who would have made valuable members of the review team.
I am seeing a process that almost could not have been designed better to make people with ME/CFS familiar with the issues with GET be frustrated and despairing. I have no doubt that our expressions of frustration will be turned back on us by some, to suggest that we aren't grateful when people try to help us, that we are a very difficult community and that is why so few people want to be associated with ME/CFS. But, a straight-forward evidence review that was staffed by qualified unbiased people would not be causing these problems.
I hope Hilda that you will understand what I am saying here. It's a car crash in motion. I will try to look away soon, but I'm concerned about the damage to the ME/CFS community that will be left at the end. Please just start again, aiming to make the review simpler and more robust.
edited
I'm concerned about the process that Hilda applied in order to select the sole person with ME/CFS for the review team. There are many people with ME/CFS with good skills in research analysis who would have made valuable members of the review team.
I am seeing a process that almost could not have been designed better to make people with ME/CFS familiar with the issues with GET be frustrated and despairing. I have no doubt that our expressions of frustration will be turned back on us by some, to suggest that we aren't grateful when people try to help us, that we are a very difficult community and that is why so few people want to be associated with ME/CFS. But, a straight-forward evidence review that was staffed by qualified unbiased people would not be causing these problems.
I hope Hilda that you will understand what I am saying here. It's a car crash in motion. I will try to look away soon, but I'm concerned about the damage to the ME/CFS community that will be left at the end. Please just start again, aiming to make the review simpler and more robust.
edited
Last edited:
I agree that the information we have about the patient on the review panel tells us nothing about her particular suitability for the role. I think an explanation of this surprising appointment lies in the criteria spelled out in Hilda's report:
https://community.cochrane.org/orga...ofile-reviews-pilot/cochrane-exercise-and#Two
I apologise if this has already been answered, but I would like to know from @Hilda Bastian
- who came up with these criteria,
- what role people selected to fit these criteria are expected to play in the technical process of reviewing research, including data analysis, and understanding and being able to contribute to decisions on what to conclude from the evidence including being able to explain why some data is unfit for purpose (eg the Chalder Fatigue Questionnaire).
There is only one of the above criteria which hints at a useful background, but it is so vague as to be meaningless:
'A history of interest in research on ME/CFS'
Edited.
https://community.cochrane.org/orga...ofile-reviews-pilot/cochrane-exercise-and#Two
Nothing in the criteria suggests the need for people with proven knowledge, skills and experience with in depth reviewing of clinical trials.
I apologise if this has already been answered, but I would like to know from @Hilda Bastian
- who came up with these criteria,
- what role people selected to fit these criteria are expected to play in the technical process of reviewing research, including data analysis, and understanding and being able to contribute to decisions on what to conclude from the evidence including being able to explain why some data is unfit for purpose (eg the Chalder Fatigue Questionnaire).
There is only one of the above criteria which hints at a useful background, but it is so vague as to be meaningless:
'A history of interest in research on ME/CFS'
Edited.
Last edited:
But to be clear that is a priority choice to prioritize pandemic response things.
[edit]
Cochrane is also an organization with many involved so I assume additional resources could have been found to help.
Last edited:
Thanks for hanging in there to help find a path forward from here, @Hutan. In replying, I'd also like to back to a post by @Trish that also had suggestions in there:
Of the 2 aspects that @Trish has pointed out as standing in the way of moving constructively forward, one it seems to me we should be able to resolve. That was the distress about my comment about advocacy/communication style (in general, not specific terms). I really can't see a productive way to have that conversation here, but there was a second option suggested: retracting it. I offered to delete it, but I obviously screwed up the opportunity I had there with a door opened for me, as that (and/or perhaps the way I phrased it) was not enough to count as retracting? If you could bear to hang in there on that for another try, I'd appreciate a second chance to understand what that means to you.
The other, is around, in part, whether there is a role in a systematic review for a person with the condition who isn't regarded as enough of a "citizen scientist", a term I'm not fond of, but I think it might be a fairly neutral term for what's being discussed here. (Patient expert to me is a wider term that respects and honors a variety of forms of expertise.) This one is harder, because there's a clash of cultures between us, and between some expectations here and how the Cochrane journal/community operates. It's a clash of values between us, too, about whether systematic reviewing is a purely objective process - or whether there is in fact no such thing as pure objectivity, and it's an activity done by a group of human beings, and therefore there's a social and sociological component, and that embeds in it factors that can affect its potential influence. And then there's the clash about how to discuss those issues while being respectful of a specific individual with ME/CFS - and by extension, anyone who reads on a public forum derogatory remarks about being someone who might not sound a whole lot different to themselves.
This is complicated, because the values are clashing so strongly, and this isn't the best way to have a discussion like that.
What I'm doing/setting in train at the moment: addressing the contentious aspects of the process in more detail in the methods report, and thinking and talking with people about how to address people's wholly understandable interest/concern about the consumer authors in this systematic review team. Neither of those things will show results in the next few days, but they are happening.
Of the 2 aspects that @Trish has pointed out as standing in the way of moving constructively forward, one it seems to me we should be able to resolve. That was the distress about my comment about advocacy/communication style (in general, not specific terms). I really can't see a productive way to have that conversation here, but there was a second option suggested: retracting it. I offered to delete it, but I obviously screwed up the opportunity I had there with a door opened for me, as that (and/or perhaps the way I phrased it) was not enough to count as retracting? If you could bear to hang in there on that for another try, I'd appreciate a second chance to understand what that means to you.
The other, is around, in part, whether there is a role in a systematic review for a person with the condition who isn't regarded as enough of a "citizen scientist", a term I'm not fond of, but I think it might be a fairly neutral term for what's being discussed here. (Patient expert to me is a wider term that respects and honors a variety of forms of expertise.) This one is harder, because there's a clash of cultures between us, and between some expectations here and how the Cochrane journal/community operates. It's a clash of values between us, too, about whether systematic reviewing is a purely objective process - or whether there is in fact no such thing as pure objectivity, and it's an activity done by a group of human beings, and therefore there's a social and sociological component, and that embeds in it factors that can affect its potential influence. And then there's the clash about how to discuss those issues while being respectful of a specific individual with ME/CFS - and by extension, anyone who reads on a public forum derogatory remarks about being someone who might not sound a whole lot different to themselves.
This is complicated, because the values are clashing so strongly, and this isn't the best way to have a discussion like that.
What I'm doing/setting in train at the moment: addressing the contentious aspects of the process in more detail in the methods report, and thinking and talking with people about how to address people's wholly understandable interest/concern about the consumer authors in this systematic review team. Neither of those things will show results in the next few days, but they are happening.
Thanks, @Trish - sorry, I hadn't seen this had been posted, when I clicked post reply on the post I'd been drafting.
I am going to expand on this in a methods report. But there's an answer to the "who came up with these criteria" question in the progress report:
"The new author team was decided by MOSS and PaPaS, in consultation with the Cochrane project manager and me. The IAG was not yet established at that time. We agreed on this composition for an author team of eight....Consumers, as defined by Cochrane, could be people with personal experience of ME/CFS or carers for them. We agreed on these criteria for these positions..."
Edited to amend quote of an edited post.
Last edited by a moderator:
Ariel
Senior Member (Voting Rights)
Would it be possible to have a few different patients with ME/CFS involved, to represent a diverse range of experiences? Specifically, someone who is on the severe end? It may be hard to find someone who is well enough, but this seems very important. Many of us do not recognize the lives or understand the issues important to those affected either a lot more or a lot less severely. This seems fair enough given the dramatic differences in the life of eg someone who can do part time work vs someone who is bedbound or housebound.
Sly Saint
Senior Member (Voting Rights)
if you go to 'search forum' and select 'search tags' then enter 'muirhead' you should find most of them here.
eta: the ME show podcast interview (with Gary Burgess) is a good one for 'patient experience'.
https://www.s4me.info/threads/the-me-show-series-two.7873/page-4#post-152104
Last edited:
What bothers me is that Hilda was searching online for candidates and then appointing them rather than running an open (and perferably transparant) process. If we compare this to say how the PSP recruited people by setting up an initial small committee (including the CMRC PAG representatives and @MEMarge nominated from here) and using that to advertise and operate a recuritment process. That got good candidates and this is at the heart of progressing.
Isn't this meant to be a template process for other reviews and including 'consumers' and hence it is worth reflecting on the process, how it is run, how it would be reproduced and how it compares to other processes that seek to get patients involved (such as the PSP).
Updates on this overall recruitment process could have been given over time as they were happening. Things like opening up for applications with a close date; what was being looked for for applicants (reflecting the range of skills and knowledge that would be valued). This didn't seem to happen (although I admit I've been very busy doing other stuff so it may have slipped by).
And I noticed on Hilda's report that
So I would have hoped that a project manager from Cochrane would have pushed for a better process.
One thing I think is important is this
It is really important that patients get to comment on the protocol as that is at the heart of what the review will say and whether it will be any good. I am worried about the brief comment as patients need time to digests if brief is say just a week that is insufficient time for a good response given peoples illness. I don't see why it can't be done at the same time as peer review.
I also think it is really important that comments are taken into account and coherent reasons (not just we will have to differ here like happended with comments on the previous cochrane review) should be given for the major criticisms. Also I'm concerned that putting a peer reviewed protocol out for further comment could lead to difficulties in getting changes as I can see comments 'but peer reviewers said it was fine' even though peer review is a really poor process (as people don't have time for detailed analysis).
So it would be good to have the full process layed out now in terms of proposed time scales and any back and forth that will be expected with the IAG, peer reviewers, the patient community etc. Lets have a process defined (it can change if necessary) rather than doing it in an adhoc manner.
It is really important that patients get to comment on the protocol as that is at the heart of what the review will say and whether it will be any good. I am worried about the brief comment as patients need time to digests if brief is say just a week that is insufficient time for a good response given peoples illness. I don't see why it can't be done at the same time as peer review.
I also think it is really important that comments are taken into account and coherent reasons (not just we will have to differ here like happended with comments on the previous cochrane review) should be given for the major criticisms. Also I'm concerned that putting a peer reviewed protocol out for further comment could lead to difficulties in getting changes as I can see comments 'but peer reviewers said it was fine' even though peer review is a really poor process (as people don't have time for detailed analysis).
So it would be good to have the full process layed out now in terms of proposed time scales and any back and forth that will be expected with the IAG, peer reviewers, the patient community etc. Lets have a process defined (it can change if necessary) rather than doing it in an adhoc manner.
Thanks for your post. Two was the maximum for consumers on this group for this process, with this timeline - but there is the IAG too, there will be peer reviewers, and there'll be other ways to contribute. And it's a Cochrane review, so if there's new evidence etc, it can kick into life again - at that point, if the intensity of conflict around this review has lessened and the review is normalized - which I believe is an achievable goal - then timelines and dealing with public pressure and expectations are removed from the situation, and major barriers/deterrents to being involved would be lifted.
Yes, I agree severity is important. (Someone declined the invitation to be an author on grounds of ill health, and we had an extended discussion about it.)
I agree with just about all of that - w00t!!! The only part I don't agree with is the peer review part: given the history here, the new editors are obviously keen to send out something they feel confident in, and I think that's totally understandable. It's not a reason to feel like this means it's basically done and dusted. (But you don't need to see the next report to know that recommendations of peer reviewers can be made now to cochrane.iag@gmail.com - I'll take them here, but there's less risk of falling through cracks if there's an email.)
Yes.
So maybe more controvertial in terms of peer review could S4ME be a peer reviewer as a group? I know that goes againt the normal model. I assume there would be confidentiality concerns but we could put together a group of patients to review and have a private forum to work in an collate comments (I think we did this pulling together comments for the NICE long covid guidence as it was confidential). That may take away from some individual patients as reviewers but may allow those whose health is not up to doing a full review (or who have prefered to stay anonymous in contributing to previous work) a chance to apply their limited time/energy to where they have skills.
MSEsperanza
Senior Member (Voting Rights)
(or both...) -- Thanks, Adrian. Very important point.
Yeah, you can do a lot when it's not a pandemic, in an organization going into pandemic response mode, and people suddenly home-schooling left, right, and center (which I can attest even hit grandmothers pretty hard). (As I said, there'll be recommendations in the end about options, timelines, etc.)
So the PSP recruitment happened during the pandemic (it is only UK wide) and I do realize the issues with working with the pandemic. Its not unusual for me to spend all day on zoom and then have to spend my evenings and weekends catching up with work (which is what I should be doing now!).
But if this is meant to be a template for the future it is really important to think through process and what is robust and repeatable.