1. Guest, the 'News in Brief' for the week beginning 23rd November 2020 is here.
    Dismiss Notice
  2. Welcome! To read the Core Purpose and Values of our forum, click here.
    Dismiss Notice
  3. Guest, NICE have published their draft guideline for ME/CFS, click here to read about it.
    Dismiss Notice

Real-world studies no substitute for RCTs in establishing efficacy (2019)- Gerstein McMurray Holman

Discussion in 'Health News and Research unrelated to ME/CFS' started by Sly Saint, Jan 23, 2019.

  1. Sly Saint

    Sly Saint Senior Member (Voting Rights)

    Messages:
    5,367
    Likes Received:
    44,552
    Location:
    UK
    Real-world studies no substitute for RCTs in establishing efficacy
    January 19, 2019

    https://www.thelancet.com/journals/...&hss_channel=tw-27013292#.XEgvxfcH2L0.twitter



    full paper here:
    http://sci-hub.tw/https://doi.org/10.1016/S0140-6736(18)32840-X

    (retweeted by Michael Sharpe; hmm)
     
    Cheshire, Esther12, Andy and 3 others like this.
  2. rvallee

    rvallee Senior Member (Voting Rights)

    Messages:
    5,169
    Likes Received:
    41,073
    Location:
    Canada
    Has anyone ever promoted this as a replacement for RCTs? Ever? Bueller? Anyone?

    Both. All of it. This isn't a binary choice where one excludes the other. The most interesting results come from when real-world data conflict with research. It opens up the gaps in research (or in the data) and promotes new research paradigms.

    But definitely one motivation in making up a strawman argument would come from people, like Sharpe, who absolutely do not want real-world data to debunk their ideas and reveal the fraud in their own research. SAD!
     
    Lisa108, EzzieD, Keebird and 2 others like this.
  3. Jonathan Edwards

    Jonathan Edwards Senior Member (Voting Rights)

    Messages:
    7,777
    Likes Received:
    80,530
    The paper about real-life data that was flagged up a few days back talked of 'too much weight' being put on RCTs rather than real-life data. That reveals the false analogy with a weighing machine and balance or with a weighted sum. More weight to one thing suggests less weight to the other.

    So the implication was there even if covert.

    Clearly the proper analysis is different. Science is about putting up theories and trying as many ways as you can to refute them. A convincing fall at any hurdle and your theory is out. If RCTs do not support a hypothesis it is not rescued by some helpful real-life data, unless methodology is in doubt. If methodology was in doubt it was never a gold standard anyway and if it is doubt in both cases you are none the wiser.

    It is interesting to see that this stuff is being aired at the moment. My impression is that the little bubble of PACEgate is not as peripheral to the general consciousness as one might have thought.
     
    rvallee, EzzieD, TrixieStix and 4 others like this.
  4. Jonathan Edwards

    Jonathan Edwards Senior Member (Voting Rights)

    Messages:
    7,777
    Likes Received:
    80,530
    The main message of the paper here seems sound but the last bit looks as if there may be a hidden agenda. There is talk of 'real-life RCTs' being easier and cheaper. There is no mention of blinding in the paper - which is the main source of cost in RCTs (it hugely complicates the administration). Maybe some people are indicating their approval because they too like the idea of cheap and cheerful trials - maybe what have been called 'pragmatic trials' - sound familiar?
     
    rvallee, EzzieD, TrixieStix and 6 others like this.
  5. Snowdrop

    Snowdrop Senior Member (Voting Rights)

    Messages:
    1,570
    Likes Received:
    10,462
    As a counterpoint there is this paper: https://www.tandfonline.com/doi/full/10.1080/07853890.2018.1453233

    "Why all randomised controlled trials produce biased results" Krauss 2018

    Seems like what has been seen as the gold standard (RCT) is coming under some scrutiny and there are some who are fighting to maintain the status quo and business as usual.

    I don't know anything on this subject but even if RCT are the epitome of science research their limitations and possible abuse ought to be a part of every researchers understanding and there should be gatekeepers to monitor those issues properly.
     
  6. Jonathan Edwards

    Jonathan Edwards Senior Member (Voting Rights)

    Messages:
    7,777
    Likes Received:
    80,530
    Indeed but that is taken as read. We should al know the limitations - like unblinded trials with subjective endpoints being useless even if randomised, or that 'controlled' has to be adequately controlled and so on.

    But the argument here is that bicycles with front and back brakes and a bell are not perfect so why not have ten bikes with no brakes. It is just a political ploy to allow shoddy evidence making.

    The gold standard is an adequately controlled randomised double blind trial. Kraus's claim that 'RCTs are not so good' is a con trick - you need the other bits as well. He seems to assume nobody knows what every trainee should know about trials - maybe because he didn't. He almost certainly has some other axe to grind.
     
    rvallee, EzzieD, TrixieStix and 2 others like this.
  7. alex3619

    alex3619 Senior Member (Voting Rights)

    Messages:
    1,412
    Likes Received:
    11,009
    Good researchers understand this. Too many do not. Journalists typically do not, but there are exceptions. The general public is usually waiting for the media to decide.
     
    rvallee likes this.
  8. alex3619

    alex3619 Senior Member (Voting Rights)

    Messages:
    1,412
    Likes Received:
    11,009
    The gold standard being the best we have, if done right, but gold standards are usually not perfect.
     
  9. duncan

    duncan Senior Member (Voting Rights)

    Messages:
    918
    Likes Received:
    3,569
    Yes. Not only are they not perfect, but their results can be "crafted", as perhaps evidenced by the three notorious Lyme RCTs. Maybe that potential is more reflective of GIGO.
     
  10. Snowdrop

    Snowdrop Senior Member (Voting Rights)

    Messages:
    1,570
    Likes Received:
    10,462
    Agreed.

    When I keep seeing this issue pop up I can't help but think about how, not so many generations ago many people (more than today - although the trend might be downward) were illiterate. I see a need for better science teaching at the grade school level so that we might have a more science literate society. With luck that might work against all of the con and scams being foisted on us in the name of science.
     

Share This Page