Risk of bias tools in systematic reviews of health interventions: an analysis of PROSPERO-registered protocols - Farrah,Young,Tunis,Zhao Nov 2019

full paper here
https://systematicreviewsjournal.biomedcentral.com/articles/10.1186/s13643-019-1172-8

 

It's really bizarre that randomized but non-controlled trials are skipped over. Is it because people just assume they don't happen? They clearly do, in fact they are the norm in BPS (and psychosomatic "research" since forever) and there has not been a single properly controlled BPS trial of ME, CFS or MUS in general, because they cannot be adequately controlled. If the numbers are remotely close to be correct about the frequency of MUS, though given the real explanation of gaps in our medical knowledge rather than unexplainable, then this affects a very sizeable share of current medical research.

This is repeated a few times in the abstract, that it is either non-randomized or a full double-blinded RCT. There are several steps in-between and they are common, in some schools of thought pretty much the only option. There are randomized open label trials that have no possibility of controls and there are also randomized open labels with inadequate controls that then go into lots of speculation about what the inadequate controls represent, things as bizarre as "maybe they've seen news coverage of this and it changed their perception", grasping at any straw in the way.

Those numbers expose a huge flaw in all published research given that meta reviews like Cochrane are held as the gold standard, that it is in fact a garbage-in-garbage-out factory. But I guess we found our non-controlled here, with "quasi" doing a lot of work that it should not be doing. No doubt the Cochrane review on GET would fall in the quasi even though they have absolutely no adequate controls and in addition to being open label are entirely subjective, another flaw that is not addressed and badly needs to. This is very flawed thinking overall.

I don't think this discussion can be had without acknowledging this and it's utterly bizarre that this seems to be skipped over as if it doesn't exist or has a limited impact on published research. I'm wondering if it's worth pointing out to the authors since it seems to be a common failure, almost as if this was taboo but in fact with the way things are going they will only be more frequently used.

That this major elephant in the room isn't even acknowledged by the people focused on talking about the room is not encouraging at all.

 

rvallee you skipped the part were Cochrane are changing their risk of bias tool to a new even more useless one . soon it will be impossible to trust anything these greedy publishing houses print .

 

What is the point of all this? If something is effective, it will prove out in properly designed trials.

This all just seems like a way to (a) get around doing trials capable of producing definitive evidence so that there is wiggle room to claim whatever is convenient by appealing to some official-looking but spurious tools or (b) use said tools to abdicate individual responsibility to critically asses research so as not to run afoul of any benefactors or colleagues.

 

It's curious that they cite Jadad et al. but don't mention the word "blind" or "blinding" once.

Jadad AR, Moore RA, Carroll D, Jenkinson C, Reynolds DJ, Gavaghan DJ, et al. Assessing the quality of reports of randomized clinical trials: is blinding necessary? Control Clin Trials. 1996;17(1):1–12.

https://www.ncbi.nlm.nih.gov/pubmed/8721797

 

so this method could be described as 'quasi modo'....(?)

eta: 'the bells, the bells'

 

It's paving the way for more BPS, which cannot stand on its own merits and needs the bar to be lowered below ground. It is largely centered around the UK, where the massive IAPT reform is also failing on its own merits and needs to be propped up because the people responsible can't accept accountability for the predictable debacle. The UK is sort of inflicting its own mistakes on the world because of historical flukes that made it a center of modern medicine.

It's also true that medicine has been running low on progress these past few years, having more to do with increasing costs of incremental progress once the easy stuff has been discovered, but the response has been instead: what if we just allow people to make stuff up about magical psychology? Most of the blame lays in the insulated approach to medicine, where patient input that can bridge the gaps is dismissed as impure, but also a plateau in technology that is bridging us from primitive understanding to all new paradigms. Unfortunately those transitions are usually confused by some people as stagnation.

It's growing pains, a sort of trial by fire attempting to bring back the worst failures of the past, a pre-science approach of merely arguing for random correlations and reversed causality, to make up for more recent failures. The end result will be to spectacularly implode the entire BPS ideology and its peripheral belief systems, but the failure has to be brought to the full before people accept that. It's a shiny "new" toy that has already failed but the hopes are high about magical powers of the mind, about Freud having been right all along.

It's a real problem, that every bit of progress is harder than it used to, but the solution is to take the worst possible way out. The only positive is that in the end it will close the door entirely to this approach, and probably belief in psychosomatic medicine entirely, but not until it has proven itself to be spectacularly incompetent under all the spotlights. Ultimately it's showing that physicians are just as gullible and prone to magical thinking as anyone else, they just require different methods but pseudoscience is about incredible evidence, not about the precise nature of the thing that has no evidence, whether it's humours, fairies, karma or psychogenic explanations.