The 'C' in RCT

Barry

Barry

Senior Member (Voting Rights)
I know much of this is old hat, but just wanted to revisit, with a particular thought in mind.

Possibly in the wrong thread, in which case mods please feel free to relocate.

As best I understand this, though others here far better qualified:

Surely a "controlled" trial is where you control so that the only things you ultimately measure, are the effects of the intervention itself; nulling out any other unwanted but unavoidable effects within the trial. So, simplistically, if your intervention arm has influences Intervention, I, plus unwanted but unavoidable influences x, y and z, then you also run another trial arm identical but without I ... i.e. with just the unwanted influences x, y and z. And so to null out x, y and z from your results you take the differences between the two arms.

So to be clear, the only influence you want to measure the effects of, is the intervention itself. What you absolutely do not want to be measuring is any effects from participants' awareness of the intervention being administered - that would be an unwanted influence that needs to be controlled for and nulled out, as per x, y and z in the above example.

But in an open label trial you simply cannot separate out the intervention itself, from participants' awareness of the intervention; two distinctly separate influences. So, surely, you cannot control an open label trial simply by having a "control" arm that has: no intervention, an alternative intervention, or mix of interventions ... because that can only work if the participants have no awareness that that is being done to them. If the arms are not identical in all respects apart from what you want to measure, then you will not be able to know what the mix of effects is that you are measuring. People's awareness of their interventions would have to be identical across all arms, which they clearly can not be, not in the real world anyway.

So I do not understand how any of these open label trials are able to claim that they are Randomised Controlled Trials? What are they controlling for? Surely it is only legitimate to claim to be controlling things that ensure you end up measuring only the intervention itself? But if intervention-awareness cannot be controlled for, then you surely cannot claim your trial to be controlled? And just because you cannot do it, is no excuse for then claiming it anyway.

In the infamous FINE trial for example, it says "Our control condition is treatment as usual by the general practitioner (GP)." https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1456982/. But this is like running a drug trial with the placebo tablet being labelled "DO NOT EXPECT ME TO WORK, I AM NOT WHAT THIS TRIAL IS ABOUT", and the intervention arms labelled the opposite; instead of being indistinguishable from each other.

So are there any regulations on what controls a trial has to have in place, in order for it to claim itself as being a controlled trial?

It feels to me there should be a standard check list that every trial protocol should have to fill out, listing all sources of biases that trials are known to be prone to, with each item being filled in with how the trial will control, or not control, for that particular bias.

And of course the 2011 PACE paper ended up not claiming it to a be a controlled trial, though its protocol - which it inevitably referenced of course - did claim it would be.

It just feels wrong that such revered terminology can be abused so easily.
 
Yes!

The principal of a "control" in science is ceteris paribus. (all other things being equal).

If the participants can tell a difference between which group they are in, then the trial is not controlled.

I note this specifically, as one study that claimed to have "blinded" participants during a non-pharmacological therapy asked participants whether they were in the "control" group or the intervention group. Almost all participants in the "intervention group" guessed incorrectly, which is to say they thought they were in (what the authors regarded as) the control group. Yet both groups involved what could be considered interventions. Hence the trial was not actually controlled at all, but somehow the authors spun this as effective blinding as the participants guessed incorrectly. To me, the blinding can only be considered to be maintained if the participants cannot tell any difference at all between the therapies, which is to say the guesses of which group they are in should be no different from tossing a series of coins.

This was a classic example of those running trials not understanding why blinding is necessary in the first place.

Studies for which blinding is not possible or is not maintained should be referred to as randomised comparison group trials, not randomised controlled trials.

I've wanted to publish something pointing this out in a major medical journal, but since I'm a nobody, they'll likely just ignore me.
 
Snow Leopard said:
If the participants can tell a difference between which group they are in, then the trial is not controlled.
Click to expand...
Even that can, at least in some cases, be adequately controlled for by allowing enough time for the placebo and other confounding effects to wash out (regress to the mean). That is what happened in PACE after 2.5 years.

Though obviously it is still much better to eliminate confounders from the start.
 
Sean said:
Even that can, at least in some cases, be adequately controlled for by allowing enough time for the placebo and other confounding effects to wash out (regress to the mean). That is what happened in PACE after 2.5 years.
Click to expand...

No. Long term followups are even less controlled as you cannot control what participants do after the trial.

The fact that some of the response biases diminish long-term is another topic altogether.
 
You are absolutely right, Barry.

AsI have probably said before, part of the problems a bad habit that philosophers are often guilty of, despite it being pointed out by the philosopher Wittgenstein - that they were taking language on holiday.

The problem is the assumption that language works like lego. But it doesn't. To be bathing does not often mean having a bath. So a controlled trial is not a trial with a control. A 'control' is any comparator that is made use of in the hope of bringing to light spurious contextual factors. It may be a negative control or a deliberately positive control.

I am fairly sure that nobody, as you say, has sat down and spelt out what is meant by a controlled trial, but to be of any use as a term it must mean a trial with enough controls to cover every plausible x y and z.

But likes many things in research, it improbably not helpful to try to define rules for how to do that. Because the whole point is to cover every possible x, y and z in every different situation. Nobody defines what a sport is because tomorrow someone will invent sport unlike any other. Tomorrow someone will want to do a trial where the x, y and z have never had to be considered before.
 
Jonathan Edwards said:
But likes many things in research, it improbably not helpful to try to define rules for how to do that. Because the whole point is to cover every possible x, y and z in every different situation. Nobody defines what a sport is because tomorrow someone will invent sport unlike any other. Tomorrow someone will want to do a trial where the x, y and z have never had to be considered before.
Click to expand...
Yes, when I said "listing all sources of biases that trials are known to be prone to", I really should have said "listing all types of biases that trials are known to be prone to".

My thought was not to try and kill off any useful trial by getting over-bureaucratic, because I'm very well aware how counter-productive that can be. But I was really thinking it would be good to have some way of getting trial designers to commit, in some accountable way, that they have thought about potential sources of bias, and how they plan to cope with them. In some cases this might be as simple as saying we cannot control for this form of bias, and so the best way we can deal with it will be to report our findings honestly with this in mind once we publish; to me that could be a real step forward. Because the way things are at the moment, nothing is going to change.

Given that a protocol is about stating up front the methodology for your trial, with a view to ensuring you don't try moving the goalposts during or after the trial, it just seems "obvious" that a statement about how the investigators plan to address issues of bias should be part of that. At the very least to give some insight into whether the investigators actually understand the issues themselves or not! I also feel that the very act of being required to commit (virtual) pen to paper, might make investigators stop and think a bit more, prior to plunging headlong into their trials.
 
I think the problem is that all junior medics are taught to recognise obvious flaws in control profiles and in most of medicine they go on applying that knowledge, but there are islands within medicine where the obvious is just ignored. If PACE had landed on my desk as a member of the UCL ethics committeeI would have said we cannot pass this because it is not going to give a meaningful result.

What you asking for is like a set of rules for presidents that includes not passing laws saying you have permanent immunity from prosecution. Everyone can see that you would only break the rule if you were dodgy. But some presidents are happy to do so.

The problem is entirely a political one - waving things through because it suits someone's vested interests.
 
Jonathan Edwards said:
I am fairly sure that nobody, as you say, has sat down and spelt out what is meant by a controlled trial, but to be of any use as a term it must mean a trial with enough controls to cover every plausible x y and z.
Click to expand...
This feels very important to me, because quacks all over the place are pushing their remedies, supposedly backed up by RCTs, when in fact the trials in question are far from "controlled". They enthusiastically cite RCTs as the "gold standard" of medical trials, which has a bitter irony to it, given there is no standard to which such trials have to abide by. So they get away with their lies because the system actively supports them in doing that.

Edit: Crossed in post with @Jonathan Edwards' post.
 
Thinking about it I guess the key issue is the politics of journal editing. The two most prominent editors in the UK seem to have node about basic quality assurance.Maybe because they were both career journalists with a medical degree rather than respected physicians whomever later into editing.Maybe the old system was as bad.
 
it seems to me one of the issues with trials and RCTs and so on is that of vested interest in the result.

There seems too much interest in a specifc outcome instead of interest in having questions answered, learning more about diseases and biological processes and raising new questions that we hadn't considered before.

Also that the result itself is an endpoint instead of just a milestone. In an ideal world the results, methodology and all anonymised data should be constructively discussed. So it adds to the pool of knowledge and investigators get the benefit of the insights of others.

Perhaps this is too idealistic?

The journals and the likes of Cochrane seem to be shoring up a system that seems more interested in maintaining researcher status rather than encouraging scientific rigour, curiosity and constructive conversation.
 
I still think that if there is no binding definition of what a Controlled trial actually is, then it is open season for the snake oil brigade to claim their bullshit offerings to be evidenced by RCTs. Pretty much any charlatan can point to some dubiously-claimed "RCT" that is in fact nothing of the sort. If there is no way to stop investigators claiming their trials to be controlled when in fact they are nothing of the sort, then it is hardly surprising people abuse that.

It's a bit like the sale of goods and services act. The service you provide has to meet a minimum standard. And where human safety is involved, it most certainly should not be a free for all. When people can be claiming to run RCTs that are not, and the knock on from that is that charlatans can claim their snake oil treatments are evidenced by "gold standard" RCTs, and when the knock on from all that is for huge numbers of people to have their lives totally f**ked up when it could have been avoided, or at least attenuated ... I just find it hard to credit that the medical trials side of things seems to have escaped closer scrutiny for so long. It just totally p**ses me off I'm afraid.

Needless to say I'm not talking at all about people like @Jonathan Edwards and the very many other good guys I am confident must be running well designed and executed trials. But there must be many others, some who aspire to be good guys but short on competence, and others who just don't give a damn. And just like everything else in life, the bad'uns mean there has to be tighter regulation for everyone; goodwill never really cuts it unfortunately. I mean ... how much longer can we really let this abuse of power (because that is what it ultimately is) go on for?

End of rant ... for the moment ...
 
This abuse of language has annoyed me to no end for years. What I see mainly is the abuse of the ambiguity over two possible uses of the c-word: controlled vs. clinical. They are seemingly used interchangeably whenever convenient to give more authority to something that deserves none and they will adapt the meaning based on the targeted audience.

When caught with this language play by people who actually care about such abuse of language as a means to an end, they merely fall back to claiming it means clinical trial and that's how it works and also clinical trials are very hard and people should be handsomely praised and rewarded for ever trying to run one because have you ever run a clinical trial it's really hard and everything. I see that thought-terminating cliché so often, this praise over how people should basically be gifted knighthoods for the simple fact of running a positive psychology clinical trial. Because trials are hard. Somehow that's an argument. As long as they are toxic positive psychology, of course, the act of giving people false hope so that they will stop complaining all the time.

As prof. Edwards pointed this is largely an editing failure. The editors know this is abuse of language but simply allow it in the space of clinical psychology, where abuse of language is not a bug, it's a feature. Just like when Wade gushed over randomized controlled trials in his SMC PR release about PACE. He knows PACE isn't controlled, the people involved at the SMC know it isn't controlled, the PACE researchers know it isn't controlled, but nobody cares to correct it because it's abuse of language and statutory authority they agree with, as a means to an end.

As it usually goes, the means simply become the ends, even more so when they are completely misaligned with reality. And so abuse of language as a means to an end has predictably lead to abuse being the only end result of this perversion of every principle of not only science but medicine as well.
 
Reading Feynman and others in physics, they designed experiments to try to disprove their hypothesis, then if they could not it was a good indication the hypothesis was correct. They also seemed happy with the idea that many years later someone could the hypothesis was wrong and it would be adjusted.

If the likes of Sharpe and Stone and the rest of them were forced to think of reasons they could be wrong and then test for those we could trust more of what they say.

For instance they could have designed a trial to see if people with ME were deconditioned instead of assuming it.

In any case, looking at things in the opposite way is a good exercise to tighten up your thinking. A brisk discussion where all the flaws in a theory were considered would be a good way for them to go.

We could help them :)
 
Mithriel said:
Reading Feynman and others in physics, they designed experiments to try to disprove their hypothesis, then if they could not it was a good indication the hypothesis was correct. They also seemed happy with the idea that many years later someone could the hypothesis was wrong and it would be adjusted.

If the likes of Sharpe and Stone and the rest of them were forced to think of reasons they could be wrong and then test for those we could trust more of what they say.

For instance they could have designed a trial to see if people with ME were deconditioned instead of assuming it.

In any case, looking at things in the opposite way is a good exercise to tighten up your thinking. A brisk discussion where all the flaws in a theory were considered would be a good way for them to go.

We could help them :)
Click to expand...

In Karl Popper’s understanding of the word, the BPS crews’ approach to research is not ‘science’. Expressed very crudely ‘science’ can never prove something, there are always alternative interpretations, it can only disprove things. So you can only have what is currently the best hypothesis, but there will only be progress if you under take research disproving that hypothesis, ending up not with a final answer, but just a better hypothesis.

My undergraduate tutor was a very eminent developmental psychologist who had initially trained under Piaget, he was incredibly bright, but his life’s work was essentially experimentally demonstrating why Piaget was wrong.

So what we are seeing with the BPS crew in relation to ME and possibly psychiatric/psychological clinical research in general is not science in this sense but a mixture of PR and advertising, where research is not looking for contradictory evidence, but designed to confirm the approach being marketed.
 
Jonathan Edwards said:
Thinking about it I guess the key issue is the politics of journal editing. The two most prominent editors in the UK seem to have node about basic quality assurance.Maybe because they were both career journalists with a medical degree rather than respected physicians whomever later into editing.Maybe the old system was as bad.
Click to expand...

They have both doubled down on this field of research while pontificating about issues with bad science.
 
dave30th said:
They have both doubled down on this field of research while pontificating about issues with bad science.
Click to expand...

And yet they could so easily have left it alone as an issue of little scientific importance in medicine as a whole.
The importance must have been the threat to medical authority posed by patients.
But the Wizards of Oz have now met the Tin Man and Dorothy.
 
You must log in or register to reply here.
Back
Top Bottom