Metaresearch on patient-reported outcomes in trial protocols and results publications suggested large outcome reporting bias, 2025, Heravi, Busse et

[Wyva]

Ala Taji Heravi ∙ Dmitry Gryaznov ∙ Jason W. Busse ∙ Christof Manuel Schönenberger ∙ Belinda von Niederhäuserne ∙ Lena Hausheer ∙ Manuela Covino ∙ Johannes M. Schwenke ∙ Selina Epp ∙ Alexandra Griessbach ∙ Malena Chiaborelli ∙ Arnav Agarwal ∙ Szimonetta Lohner ∙ Julian Hirt ∙ Stefan Schandelmaier ∙ Simon B. Egli ∙ Moshao Amos Makhele ∙ Alain Amstutz ∙ Dominik Mertz ∙ Anette Blümle ∙ Erik von Elm ∙ Ramon Saccilotto ∙ Ayodele Odutayo ∙ Sally Hopewell ∙ Benjamin Speich ∙ Matthias Briel - On behalf of the Adherence to SPIRIT Recommendations (ASPIRE) Study Group

Highlights​

• One-third of the trial protocols did not consider patient-reported outcomes (PROs).
• Prevalence of PROs in protocols varied substantially across medical fields.
• In trial result publications, 21% failed to report any prespecified PROs.
• 36% of result publications reported more, less, or different PROs than protocols.
• The reporting quality of PROs in trial publications was often deficient.

Abstract​

Objectives​

Patient-reported outcomes (PROs) provide crucial information for evaluating health-care interventions, but previous research in specific disease areas suggested infrequent use and incomplete reporting of PROs. We examined the prevalence and characteristics of PROs in randomized clinical trial (RCT) protocols across medical fields, their reporting quality, and the consistency between PROs specified in trial protocols and subsequent reporting in trial publications.

Study Design and Setting​

We included 237 RCT protocols approved in 2012 and 251 approved in 2016, by ethics committees in Switzerland, Germany, and Canada. We systematically searched for corresponding peer-reviewed results publications and results on trial registries. Pairs of reviewers independently extracted characteristics of RCT protocols, PROs specified in protocols and reported in corresponding results publications, and assessed the reporting quality of RCTs with a PRO as the primary outcome using the Consolidated Standards of Reporting Trials–patient-reported outcome (CONSORTs-PRO) extension.

Results​

Out of 488 included RCT protocols, 147 (30%) did not report use of a PRO; 97 (20%) specified a PRO as the primary outcome and an additional 244 (50%) as a secondary outcome. The prevalence of PROs varied substantially across medical fields, ranging from 100% in rheumatology and psychiatry to about one-third in cardiology and anesthesiology. At 8–10 years after RCT approval, results were available for 264 of the 341 (77%) trial protocols that prespecified PROs. Forty-four percent of the published trials (115/264) reported all PROs as defined in the protocol, 21% (55/264) did not report any prespecified PROs, and 36% (94/264) reported more, fewer, or different PROs than those prespecified. These findings were consistent between trial protocols approved in 2012 and 2016. Among 63 peer-reviewed RCT publications that reported a PRO as their primary outcome, reporting quality was often inadequate, with seven of 13 CONSORT-PRO items.

Conclusion​

Less than half of RCT protocols with planned PROs reported them as specified in corresponding published results, suggesting outcome reporting bias, and PRO reporting quality was often deficient. These limitations complicate informed decision-making between patients and health-care providers, as well as the development of evidence-based clinical practice guidelines.


What is new?

Key findings
• The prevalence of patient-reported outcomes (PROs) in clinical trials varied substantially across medical fields.
• Among corresponding result publications, only 44% reported PROs as defined in their protocols, 21% failed to report any prespecified PROs, and 36% reported more, less, or different PROs than prespecified in the protocol.
• Reporting quality of PROs in result publications of trials was often deficient.

What this adds to what was known?
• PRO reporting needs to become more reliable across medical fields.

What is the implication and what should change now?
• Investigators should improve PRO reporting, and ethics committees, funding agencies, and journal editors should enforce the use of reporting guidelines for PROs.

Open access: https://www.jclinepi.com/article/S0895-4356(25)00155-6/fulltext

 

[Jonathan Edwards]

So this shows that trials are not just done badly, but inconsistently badly.
The thought police would prefer it to be consistently badly?

 

[Hutan]

The thought police would prefer it to be consistently badly?

Although a relevant patient reported outcome here or there, measured as planned in the protocol, in a blinded study or as an adjunct to objectively measured outcomes in an unblinded study would of course be fine.

At 8–10 years after RCT approval, results were available for 264 of the 341 (77%) trial protocols that prespecified PROs.

Never mind the PRO reporting, just a report on a trial within a reasonable timeframe seems to be an unachievable aspiration for many researchers. With a time period of 2012 to 2016, there was no pandemic to blame. That's a lot of results stuffed in the bottom drawer somewhere.

Jason Busse - for those not familiar with him, googling him brings up a whole lot of words that ring alarm bells - Gordon Guyatt, McMaster University, chiropractic, chronic pain, chronic fatigue, medically unexplained syndromes, Cochrane. There's a lot of irony in Busse et al telling others how to do research.

 

[Trish]

I'm interested to know what they think of the reverse problem where trials use PROMs and objective measures, but only report the PROM results because the objective results inconveniently don't support the researchers' prejudices. That's what we are plagued with from Busse's BPS friends.

 

[rvallee]

Investigators should improve PRO reporting, and ethics committees, funding agencies, and journal editors should enforce the use of reporting guidelines for PROs.

None of this makes any difference because even accounting for all of this, almost all trials where PROs are claimed to be used aren't actually patient-reported outcomes, they are clinician-interpreted derivations of PROs, where most of the flaws are in the design, all those flaws carry into not just the interpretation but the reporting of data. And they distort things even further when evangelizing them, the PACE gang knew full well that headlines about how "exercise cures chronic fatigue" and featured them vaguely saying the same were as dishonest as "Tylenol causes autism" and they never corrected them because this is the lie they wanted the world to believe in.

What is systematically found is that there is a huge gap between what patients report and what researchers "find" and clinicians "observe". That gap is entirely out of the numerous biases and oddities of a system that is clearly built to produce the false appearance of achievement, and as a result has become completely unfocused, even uninterested, in doing meaningful work. They are trying to fix this gap by having fake PROs that make it seem as if we judge their work as competent and useful.

Because technically awful questionnaires like the Chalder Fatigue Questionnaire is a PRO. Except it's not a real PRO, it's more like the tipping options at a restaurant, where less than 25% is not even an option. All the choices have been made for us, all we can do is choose between the least worse options, none of which ever represent reality, simply because they don't accept reality as valid.

And every time objective outcomes don't match subjective ones what do they do? Blame our perception. Somehow. Using flawed "PROs". If we're not active enough, with actigraphy, they'll still say we underestimate our ability to function, or whatever.

And even when everything else works out, they still don't know how to interpret them because they know nothing of our subjective experience, and so have nothing to compare to and validate them. We keep seeing this thing where psychological studies "validate" bad questionnaires by comparing them to other bad questionnaires. None of this has anything to do with outcomes, patient-reported or otherwise. It's entirely about creating appearances for them, never for us. Because without this distortion they pretty much have accomplished nothing.

There's a lot of irony in Busse et al telling others how to do research.

This is one reason why even judging their work based on their intent does not improve anything. Because their intent has nothing to do with helping us, it is literally impossible to help people by doing all the wrong things along the way. All they ever want is this:

The thought police would prefer it to be consistently badly?

And this is always bad for us. Hence their intent to help us is not the least bit credible, because this is exactly how not to help anyone. What they do is like "gifting" something I want to someone I know doesn't want it, and am "so happy" to take it off their hands since they won't be using it, but "it's the thought that counts", and it sure does, just not the way they want it.

 

[Creekside]

They should check to see how often the decision to include PROs was made at the start of the study, rather than after, when it was known whether it would make the results look better or worse.

As rvallee pointed out, the value of PROs is questionable and inconsistent. How positive the patients reports are depends strongly on how manipulative the researchers are. I doubt that any guidelines will improve the situation.

 

[rvallee]

As rvallee pointed out, the value of PROs is questionable and inconsistent. How positive the patients reports are depends strongly on how manipulative the researchers are. I doubt that any guidelines will improve the situation.

I can't really see that changing while it has become standard for 'trialists' to 'trial' their own creation. Almost all of the biopsychosocial treatments are designed, implemented and tested by the same people. Which barely changes anything when they all pretty much copy-paste the same stuff, but they still have a lot of stakes in the outcome. They 'test' their own thing, grade their own homework. The levels of bias in this discipline are extreme because they want them this way when they grade their own homework.

So they will always be at least very biased, but even being very manipulative has become standard, so implementing real standards would basically kill their whole circus. Nothing they do can stand an actual rigorous assessment, precisely because they design everything with the full awareness that it doesn't even matter: everyone cheats anyway.

 

[Creekside]

Patient-reported outcomes (PROs) provide crucial information for evaluating health-care interventions,

They can possibly provide crucial information. I can imagine scenarios where that's true. If 95% of patients report vomiting after 3 days on a drug, that's valuable data. However, that's for easily verifiable outcomes. When it's for non-verifiable outcomes ("feeling better"), it's just anecdotal evidence, easily manipulated for a desired result.

I can imagine some sort of guidelines being useful in specific situations, but it's more likely that a committee with no real understanding of reality will come up with general guidelines that will be abused rather than used. If the guidelines did work for one specific study, maybe one with verifiable results, researchers with bad studies would use that to justify their garbage. So, official guidelines could do more harm than good.