Caroline Struthers' letter to Cochrane Governing Board

I think we should pretty much assume that no help will come from the gatekeepers of medicine. It shouldn't stop anyone from trying, at least it gets it all on record and will matter in the long term, but Tovey's responses were basically a politician's non-responses. It's simply not a serious process because they don't think it needs to be. They only use the tools and institutions of medicine to give them the authority to do what they want, not because they think it's medically relevant. All the power of medical authority with none of the accountability, a classic recipe for disaster.

Medicine needs massive reform for patients' rights. There is no due process, no ability to appeal, no evidence is necessary to convict. Evidence can be withheld or arbitrarily dismissed. It can even be asserted without evidence based on eminence. And here we have clear abuse of authority leading to a massive human rights failure because the standards and ethics of medicine can be waived off at will if the majority opinion is "I just don't believe in these new mumbo jumbo diagnoses".

These are supposed to be the people helping us. They are doing exactly the opposite, acting as oppressive forces denying us basic human rights. We don't have a messaging problem, we have a sabotage-from-within problem.

 

We patients currently have no political power, so those with power do with us as they please.

One solution would be to acquire political power. That is much easier if we worked with other patients who are in a similar situation. Then we could create a funding system where patient representatives can prevent further funding of BPS garbage research.

 

I think Tovey was beginning to understand how problematic it would get for Cochrane to quadruple-check that they stand behind this research and how it will play out when the house of cards tumbles. Their credibility will take a huge shot, the failure is on a scale that they don't really comprehend yet but they know it will be bad.

But then Cochrane was reminded how much political influence can hurt them. When the former head of the RCGP basically calls you a wet rag in public, knowing her influence, by herself and in addition to her rotten husband, then you perk up and listen hard. The Reuters hit job was a loud shot to the bow and the message was received and digested.

Political influence made Tovey cave. This is as clear as it gets. He tried to mitigate the damage but his unhelpful beliefs about his "independence" were corrected.

 

Cochrane seems to enjoy the top spot right now as far as being taken seriously. Until there is a paradigm shift this won't hurt their standing so they're resting on their yannis.

Of course with hindsight this will look catastrophically bad but most medical professionals dislike us anyway so for now we can be entirely shut out, even harmed, without consequences. The SMC-Reuters stunt probably had more influence than our decades of advocacy and all the biomedical research. Politics control this issue, not science, not our welfare and especially not our rights.

 

Yeah - it made me think of a civil servant's attempt to avoid the real issues.

 

I think a lot of psychiatric conditions are about people having distorted self perceptions. So psychs live in a world where the way to cure their patients is to change their self perceptions. And the way they assess their patients' progress is to ask them questions which reveal their prevailing self perceptions. So far as that goes, there is a lot of validity in that, though patients are hopefully also asked more objective questions as well. e.g. If someone is acutely introverted and withdrawn, then checking if they are getting out and meeting people might be a more objective progress check, together with subjective questions.

It may also be true I think, that for people whose psychiatric condition is about self perceptions, simply knowing what treatment you are getting, and how good it's reputed to be, could very likely positively bias your self perceptions; the bias forms part of the treatment, and validly so! Anything that positively influences the person's self perceptions, is a potential treatment component.

So for many psychiatrists the combination of subjective outcomes with unblinded treatments is probably not just the norm, but prerequisite; they are probably surprised at all the fuss. I also wonder how this approach serves their patients whose problems are not rooted in self perception issues.

Their biggest mistake has been to gate crash into the realm of physical illnesses, whilst unthinkingly presuming their belief system maps onto it, and believing it to be science.

That, I believe, is the universe many psychiatrist exist within, and for them it is their science; cannot get their heads round anything else. And I think they carry an awful lot of influential people along with them, including people with less scientific training and rigour, and therefore incapable of distinguishing between something being fact rather than opinion.

So I think this is what we are up against, and have to overcome. And the first step to solving a problem is understanding the problem.

I find it somewhat ironic that the problem we face is one of changing the distorted perceptions, of those belonging to a discipline which includes in its remit, curing people of having distorted perceptions.

 

I was going to include a reference to the Cochrane Handbook in my blog (thanks for the kind comments btw!), but changed my mind. In fact, I now realise the quotes I was going to use came from the AHRQ (US) systematic review guidance, which mentions the Cochrane Handbook, rather than the CH itself.

In particular, in Table 2 - Taxonomy of core biases in the Cochrane Handbook, they deliniate each of the key biases.

For example: Performance bias

However, I've just checked the Cochrane handbook (2012 version) and bias toolkit, and they only mention blinding under performance bias, and not contamination, unbalanced provision, or differences.

Then for detection bias:

But this quote in the AHRQ guidance stood out to me:

I am looking for a similar passage in the Cochrane Handbook, because it really ought to be there - and if it isn't, then that really is "terrible"!

[The Cochrane Handbook is a devil to search for anything though - aargh!]

 

Found it! (From Cochrane Handbook v 5.1)

 

Thats just a matter of opinion!

 

So what's the correct rating for the PACE trial?

Er, not to ask you to do a whole review on your own in the next 2 minutes... but whether we can downgrade it by 2 levels?

 

There are additional problems with PACE, including use of the Oxford criteria (no studies based on the Oxford criteria are relevant to ME/CFS), and including outcome switching, sending a pro GET/CBT newsletter mid study to participants that represent research malpractice all of which surely require excluding PACE data from the analysis befor you get to any rating of susceptibility to bias.

Any reasonable review of excerise data in ME/CFS should exclude all studies using the Oxford criteria and also exclude any demonstrating research malpractice (ie PACE) which should not even have been published in any respectable scientific journal, before you even get to assessing the reliability of the data included.

In this review I don't think anything remains that can reliably be said to relate to ME/CFS just to the symptom of 'chronic fatigue'. This is a matter of objective truth not just opinion.

If the review was rewritten as the use of exercise in patients with 'chronic fatigue' (not ME/CFS) there would be some studies left to include other than PACE, but surely even then the use of subjective measures in unblinded trials should give its findings the lowest level of reliability and it should include the disappearance of any effects on long term follow up and an analysis of the null results for objective data when available.

 

I'm probably wishing for the moon but I still think that a different group made up of independant clinicians/scientists and patients should and could submit a better, upto date, review.

eta punctuation

 

It's not a randomised controlled trial, so it shouldn't have been included in the first place. The same goes for pretty much every study in the review.

 

Failure to follow the protocol is the least of its problems.

The thing about protocols and guidelines and toolkits and checklists is that they are there to enable comparability, to make sure that things are reported in the same way. They are not there to guarantee the quality of a study or to ensure that it is done properly (although they might indicate problems to be investigated if not followed correctly). I see this time and time again, and in many different circumstances, that folks think that just because they "followed the protocol", that everything is fine. It isn't. And it certainly isn't if the protocol was flawed to start with.

However, making changes to a flawed protocol to make it less flawed should be a good thing. And in some senses the PACE authors did that, by listening to some of the criticism levelled at them after they published the trial protocol. However, not all the changes they made were good - for example, changes to the outcome thresholds. And of course, publishing a protocol 3 years into the study is utterly useless if the flaws are more substantial - like using subjective endpoints in an unblinded study - there's no way you can undo that!

The problem with the Cochrane protocol was that it was based on the flawed protocols of all the studies included, so it simply replicated them. It then failed to adequately compare the studies included. The most serious problem was the failure to scrutinise the use of the measures (particularly the CFQ), and to assume that they even could be combined across completely different trial populations. The need to split the analyses into bimodal and likert(ish), and the different results they got as a result, should have alerted them to that.

 

Does the Cochrane Handbook say anywhere that trials which aren’t randomised controlled trials shouldn’t be included in their reviews?

 

Sorry, I realised I could look this up myself.

I found this in the Cochrane Handbook (version 5.1):

The above criteria does not appear to exclude uncontrolled randomised trials. Is this a flaw of the Cochrane eligibility criteria?

(I’m feeling a bit out of my depth in this discussion so I hope I’m not asking stupid questions!)

Thanks by the way @Lucibee for the great blog post.

 

Not necessarily for systematic reviews in general, but they were doing a meta-analysis and combining results from lots of different studies, which is going to be really tricky to interpret unless there is some element of control between groups and within and between studies (even if it's not as far as placebo control).

In the review, they didn't seem to provide any sort of control over the different types of GET used, how long the treatment lasted, how intensive it was, whether it was GET or GAT.

Again, it's not so much about rules or criteria - it's about their understanding of what they are doing and why, and whether they can justify that.

I do think this is a problematic statement though. Just because a trial is randomised, doesn't necessarily make it rigorous, and yet that's what folks tend to believe. Look! It's a randomised trial! Everything it says must be true!

There are plenty of ways to completely mess up a trial after you've done the randomisation bit.