Real-world studies no substitute for RCTs in establishing efficacy (2019)- Gerstein McMurray Holman

Real-world studies no substitute for RCTs in establishing efficacy
January 19, 2019

https://www.thelancet.com/journals/...&hss_channel=tw-27013292#.XEgvxfcH2L0.twitter



full paper here:
http://sci-hub.tw/https://doi.org/10.1016/S0140-6736(18)32840-X

(retweeted by Michael Sharpe; hmm)

 

Has anyone ever promoted this as a replacement for RCTs? Ever? Bueller? Anyone?

Both. All of it. This isn't a binary choice where one excludes the other. The most interesting results come from when real-world data conflict with research. It opens up the gaps in research (or in the data) and promotes new research paradigms.

But definitely one motivation in making up a strawman argument would come from people, like Sharpe, who absolutely do not want real-world data to debunk their ideas and reveal the fraud in their own research. SAD!

 

The paper about real-life data that was flagged up a few days back talked of 'too much weight' being put on RCTs rather than real-life data. That reveals the false analogy with a weighing machine and balance or with a weighted sum. More weight to one thing suggests less weight to the other.

So the implication was there even if covert.

Clearly the proper analysis is different. Science is about putting up theories and trying as many ways as you can to refute them. A convincing fall at any hurdle and your theory is out. If RCTs do not support a hypothesis it is not rescued by some helpful real-life data, unless methodology is in doubt. If methodology was in doubt it was never a gold standard anyway and if it is doubt in both cases you are none the wiser.

It is interesting to see that this stuff is being aired at the moment. My impression is that the little bubble of PACEgate is not as peripheral to the general consciousness as one might have thought.

 

The main message of the paper here seems sound but the last bit looks as if there may be a hidden agenda. There is talk of 'real-life RCTs' being easier and cheaper. There is no mention of blinding in the paper - which is the main source of cost in RCTs (it hugely complicates the administration). Maybe some people are indicating their approval because they too like the idea of cheap and cheerful trials - maybe what have been called 'pragmatic trials' - sound familiar?

 

As a counterpoint there is this paper: https://www.tandfonline.com/doi/full/10.1080/07853890.2018.1453233

"Why all randomised controlled trials produce biased results" Krauss 2018

Seems like what has been seen as the gold standard (RCT) is coming under some scrutiny and there are some who are fighting to maintain the status quo and business as usual.

I don't know anything on this subject but even if RCT are the epitome of science research their limitations and possible abuse ought to be a part of every researchers understanding and there should be gatekeepers to monitor those issues properly.

 

Indeed but that is taken as read. We should al know the limitations - like unblinded trials with subjective endpoints being useless even if randomised, or that 'controlled' has to be adequately controlled and so on.

But the argument here is that bicycles with front and back brakes and a bell are not perfect so why not have ten bikes with no brakes. It is just a political ploy to allow shoddy evidence making.

The gold standard is an adequately controlled randomised double blind trial. Kraus's claim that 'RCTs are not so good' is a con trick - you need the other bits as well. He seems to assume nobody knows what every trainee should know about trials - maybe because he didn't. He almost certainly has some other axe to grind.

 

Good researchers understand this. Too many do not. Journalists typically do not, but there are exceptions. The general public is usually waiting for the media to decide.

 

The gold standard being the best we have, if done right, but gold standards are usually not perfect.

 

Yes. Not only are they not perfect, but their results can be "crafted", as perhaps evidenced by the three notorious Lyme RCTs. Maybe that potential is more reflective of GIGO.

 

Agreed.

When I keep seeing this issue pop up I can't help but think about how, not so many generations ago many people (more than today - although the trend might be downward) were illiterate. I see a need for better science teaching at the grade school level so that we might have a more science literate society. With luck that might work against all of the con and scams being foisted on us in the name of science.